ComputeMode on top of Debian Squeeze: step by step process

This page describes the installation step by step of ComputeMode server on top of a Debian Squeeze system.

	Note
	Pre-built ComputeMode server appliances are also available for a quick setup of ComputeMode within a virtual machine

If you don't want to use a Virtual appliance for your Computemode server, you can read and follow this part to install from scratch a new server.

Install the base Debian Squeeze system

• boot Debian Lenny install media (netboot cdrom for instance), i386 or amd64 flavor, and proceed to the install
• in tasksel, select web server + dns server + standard system
• install OpenSSH server

apt-get install openssh-server

We now have a up and remotely accessible wonderful running Debian system !

Update the package sources

• Add apt sources for Computemode repositories

cat <<EOF > /etc/apt/sources.list.d/computemode.list
deb http://computemode.imag.fr/files/debian/squeeze ./
EOF

• Add apt sources for OAR repositories

cat <<EOF > /etc/apt/sources.list.d/oar.list
deb http://oar-ftp.imag.fr/oar/2.4/debian/ squeeze main
EOF

• Some options can be tweaked:

cat <<EOF > /etc/apt/apt.conf.d/000custom
APT::Default-Release "stable";
APT::Cache-Limit 40000000;
APT::Get::Purge;
Acquire::PDiffs "false";
EOF

• Finally:

apt-get update

Installation of ComputeMode web interface

We install ComputeMode packages, with the dependancies (Apache2, PHP, aso)

apt-get install cmwebadmin cmwebadmin-data cmwebadmin-manual cmwebadmin-portal

• accept deps, accept db reset, accept the sudoers modification
• get rid of the RedirectMatch ^/$ /apache2-default/ in apache default site
• remove /var/www/index.html to get the ComputeMode site working
• move /var/www/index.htm to /var/www/index.html
• sed -i 's/;suhosin.get.max_value_length = 512/suhosin.get.max_value_length = 2048/g' /etc/php5/conf.d/suhosin.ini

We have to install MDB2 module for postgresql:

apt-get install php-mdb2-driver-pgsql php-mdb2 php-net-ldap

If you're seeing an error from the apache log file with the following message: "PHP Deprecated: Assigning the return value of new by reference is deprecated in /usr/share/php/MDB2.php on line". It's a bug from the MDB2 module currently fixed in the beta version (http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=571702#20). If you want to use the beta version, just install the MDB2 module from "pear" instead of "apt-get":

pear config-set preferred_state beta
pear install mdb2 && pear install MDB2_Driver_pgsql
pear config-set preferred_state stable

You can also install tools used by cmwebadmin-portal:

apt-get install phppgadmin
apt-get install mrtg mrtg-contrib && mkdir /var/www/mrtg

and finish the installation of cmwebadmin activating the cron for the interface:

echo '* * * * *      root    /usr/bin/wget -O- -q http://localhost/cm/engine/cronautomaton.php >/dev/null' > /etc/cron.d/cronautomaton

Note: when apache is starting, you could see the following warning: "Could not reliably determine the server's fully qualified domain name". To avoid this warning just type the command:

# echo 'ServerName "localhost"' >> /etc/apache2/apache2.conf
# /etc/init.d/apache2 restart

Installation of HPA's tftp server

Accept "/srv/tftp" for TFTP root directory during the installation:

# apt-get install tftpd-hpa

Installation and configuration of the core services

• First install ISC DHCP and DNS server, as well as Linux NFS server

apt-get install bind9 isc-dhcp-server nfs-kernel-server

• Then retrieve Computemode ready to use configuration files

cd /tmp; wget http://computemode.imag.fr/files/cm-conf-files-squeeze.tgz

• We can now put the configuration files in place

tar xvfz /tmp/cm-conf-files-squeeze.tgz -C /
chown -R bind:bind /var/cache/bind

• Fix the configuration files with regard to the IP address which is actually assigned to your server and fix the DNS forwarders in /etc/bind/named.conf.options
• Then run at once (if accessing the machine remotely) the reconfiguration of the network interfaces

ifdown eth0 ;ifup eth0

• Create the directory to export for NFSROOT

mkdir /cm

• And finally restart the services

invoke-rc.d bind9 restart
invoke-rc.d isc-dhcp-server restart
exportfs -rv

• If you don't want to manage the know_hosts file for the SSH server, just un-comment the following line into the /etc/ssh/sshd_config file:

#IgnoreUserKnownHosts yes

Installation of an account database

ComputeMode diskless node image ships with NIS configured, using the "computemode" domaine. If you do want to use NIS and run the NIS server on the ComputeMode server, run:

• check that the configuration is set to act as a NIS master in /etc/default/nis (file provided in the cm-conf-files.tgz we retrieved above)

NISSERVER=master

• then

apt-get install nis

• set the domain name to "computemode"
• run the NIS domain initialization

/usr/lib/yp/ypinit -m

• create your user accounts, etc...
• and generate the NIS maps

cd /var/yp
make

• To finish restart your NIS server

invoke-rc.d nis restart

Other services/utils

Other services can be configured:

• apt-get install syslog-ng
• apt-get install ntp
• apt-get install dnsutils
• ...

Installation of the resource manager OAR

• You have to install package for Postgresql connection:

 apt-get install libdbd-pg-perl

• And then the OAR's packages:

 apt-get install oar-admin oar-common oar-doc oar-node oar-server oar-user

• Then you have two ways to create the database (extract from the OAR admin manual: http://oar.imag.fr/admins/admin_documentation.html#configuration-of-the-cluster):

First, you can manually create the database, create users and postgresql role:

 sudo su - postgres

 createuser -P
   Enter name of role to add: oar
   Enter password for new role:
   Enter it again:
   Shall the new role be a superuser? (y/n) n
   Shall the new role be allowed to create databases? (y/n) n
   Shall the new role be allowed to create more new roles? (y/n) n
   CREATE ROLE

 createuser -P
   Enter name of role to add: oar_ro
   Enter password for new role:
   Enter it again:
   Shall the new role be a superuser? (y/n) n
   Shall the new role be allowed to create databases? (y/n) n
   Shall the new role be allowed to create more new roles? (y/n) n

 createdb oar

 psql -Uoar -h127.0.0.1 oar
   \i /usr/lib/oar/pg_structure.sql
   \i /usr/lib/oar/pg_default_admission_rules.sql
   \i /usr/lib/oar/default_data.sql
   \q

 psql -Uoar -h127.0.0.1 oar
   GRANT ALL PRIVILEGES ON schema,accounting,admission_rules,assigned_resources,
   challenges,event_log_hostnames,event_logs,files,frag_jobs,gantt_jobs_predictions,
   gantt_jobs_predictions_visu,gantt_jobs_resources,gantt_jobs_resources_visu,
   job_dependencies,job_resource_descriptions,job_resource_groups,
   job_state_logs,job_types,jobs,moldable_job_descriptions,queues,
   resource_logs,resources,admission_rules_id_seq,event_logs_event_id_seq,
   files_file_id_seq,job_resource_groups_res_group_id_seq,
   job_state_logs_job_state_log_id_seq,job_types_job_type_id_seq,
   moldable_job_descriptions_moldable_id_seq,resource_logs_resource_log_id_seq,
   resources_resource_id_seq,jobs_job_id_seq TO oar;

   GRANT SELECT ON schema,accounting,admission_rules,assigned_resources,event_log_hostnames,
   event_logs,files,frag_jobs,gantt_jobs_predictions,gantt_jobs_predictions_visu,
   gantt_jobs_resources,gantt_jobs_resources_visu,job_dependencies,
   job_resource_descriptions,job_resource_groups,job_state_logs,job_types,
   jobs,moldable_job_descriptions,queues,resource_logs,resources,admission_rules_id_seq,
   event_logs_event_id_seq,files_file_id_seq,job_resource_groups_res_group_id_seq,
   job_state_logs_job_state_log_id_seq,job_types_job_type_id_seq,
   moldable_job_descriptions_moldable_id_seq,resource_logs_resource_log_id_seq,
   resources_resource_id_seq,jobs_job_id_seq TO oar_ro;
   \q

Or you can use the /usr/lib/oar/oar_psql_db_init command to automate the process:

  sudo /usr/lib/oar/oar_psql_db_init

• To complete the database installation, don't forget to configure postgresql in both case:

  sudo vi /etc/postgresql/8.4/main/pg_hba.conf

and insert those two lines:

host    oar         oar_ro            127.0.0.1          255.255.255.255    md5
host    oar         oar               127.0.0.1          255.255.255.255    md5

• Don't forget to edit the /etc/oar/oar.conf file to configure the database and account access for OAR:

 vi /etc/oar/oar.conf

For example:

 DB_TYPE="Pg"

 # DataBase hostname
 DB_HOSTNAME="localhost"

 # DataBase port
 DB_PORT="5432"

 # Database base name
 DB_BASE_NAME="oar"

 # DataBase user name
 DB_BASE_LOGIN="oar"

 # DataBase user password
 DB_BASE_PASSWD="oar"

 # DataBase read only user name
 DB_BASE_LOGIN_RO="oar_ro"

 # DataBase read only user password
 DB_BASE_PASSWD_RO="oar_ro"

Uncomment the following line into the /etc/oar/oar.conf file, to configure the ping checker in order to check automatically if a node is alive:

PINGCHECKER_TAKTUK_ARG_COMMAND="broadcast exec timeout 5 kill 9 [ oarnodecheckquery ]  

To configure the Power Saving module, just configure the following variables into the /etc/oar/oar.conf file:

 ENERGY_SAVING_INTERNAL="yes"
 ENERGY_SAVING_NODE_MANAGER_WAKE_UP_CMD="/etc/oar/wake_up_nodes.sh"
 ENERGY_SAVING_NODE_MANAGER_SLEEP_CMD="/etc/oar/shut_down_nodes.sh"
 ENERGY_SAVING_NODE_MANAGER_WAKEUP_TIMEOUT="120"
 SCHEDULER_NODE_MANAGER_IDLE_TIME="60"
 SCHEDULER_NODE_MANAGER_SLEEP_TIME="60"

then, restart the oar-server:

/etc/init.d/oar-server restart

• You also have to insert some properties into the OAR database for ComputeMode:

oarproperty -a cpu
oarproperty -a mem
oarproperty -c -a cluster
oarproperty -a core
oarproperty -c -a mac_address

• You can also install monitoring OAR tools: DrawGantt and Monika

 apt-get install oar-web-status

• and to complete the installation (from http://oar.imag.fr/admins/admin_documentation.html#visualization-tools-installation):

 apt-get install ruby libdbd-mysql-ruby libdbd-pg-ruby libgd-ruby1.8
 vi /etc/oar/monika.conf

With for example:

 hostname = localhost
 dbport = 5432
 dbtype = psql
 dbname = oar
 username = oar
 password = oar

 vi /etc/oar/drawgantt.conf

With for example:

 # Database type ("mysql" or "Pg")
 DB_TYPE: Pg

 # Database connection port (default 3306 for mysql or 5432 for Pg)
 DB_PORT: 5432

 # DataBase hostname
 DB_HOSTNAME: localhost

 # Database base name
 DB_BASE_NAME: oar

 # DataBase read only user name
 DB_BASE_LOGIN_RO: oar_ro

 # DataBase read only user password
 DB_BASE_PASSWD_RO: oar_ro

 # To have an easier view with CM's hostnames
 left_offsetgridx: 300

 #To configure Drawgantt in order to print cpu/core reservation
 # Change the two lines: second_field_property: "cpu", by:
 second_field_property: "core"
 
 # And uncomment the following line at the end of the file:
 prop_hierarchy:
  - network_address
  - cpu
  - core

And to finish:

 cp /etc/oar/drawgantt.conf /etc/oar/monika.conf /usr/lib/cgi-bin/

And configure the "sudoers" file for the computemode web site:

cat <<EOF >> /etc/sudoers
www-data ALL=(root) NOPASSWD: /usr/sbin/oarnodesetting
www-data ALL=(root) NOPASSWD: /usr/sbin/oarproperty
www-data ALL=(root) NOPASSWD: /usr/sbin/oarremoveresource
www-data ALL=(root) NOPASSWD: /usr/sbin/oaradmin
www-data ALL=(root) NOPASSWD: /usr/sbin/arping
oar ALL=(root) NOPASSWD: /usr/sbin/etherwake
EOF

Installation of the Module software

Module is a software environment mangager. If you want to install module sofwtare on you CM server, just follow thoses steps:

* apt-get install -y --force-yes tcl-dev tcllib
* cd /tmp && wget http://downloads.sourceforge.net/project/modules/Modules/modules-3.2.9/modules-3.2.9c.tar.gz && tar zxvf modules-3.2.9c.tar.gz
* cd /tmp/modules-3.2.9 && ./configure && make && make install && rm -Rf /tmp/modules*

Then to give access to the module command to your users, insert the two following lines into the $HOME/.bashrc file:

shell=`basename $SHELL`
. /usr/local/Modules/3.2.9/init/$shell

Installation of ComputeMode nodes image

• Retrieve the Debian Squeezee node image and untar it. For the 64bit mode node:

cd /tmp/; wget http://computemode.imag.fr/files/cm-node-squeeze-amd64.tar.gz
tar xvfz /tmp/cm-node-squeeze-amd64.tar.gz -C /

• For the 32bit mode node:

cd /tmp/; wget http://computemode.imag.fr/files/cm-node-squeeze-i386.tar.gz
tar xvfz /tmp/cm-node-squeeze-i386.tar.gz -C

Edit the /var/www/cm/modules/index.html file and change the '64bits' version for '32bits' version

• Configure the OAR,root, www-data access to this node and its environement:

cd /cm/debian && sh ./post-build.sh