Tag Archive for 'SMF'

Start Virtualbox VMs at boot time with SMF (2)

Published on Tuesday September 1st, 2009 in Unix & Linux. 8 Comments Tags: , , .

In my latest post, I wrote about an SMF manifest and method I developped to start and stop my Sun Virtualbox virtual machines at boot time. I explain here some changes I’ve done to both files, and why. If you were interested in it, you are  encouraged to update both files on your system (you can download the files at the end of this post).

  • Handling multiple VMs

The idea was the svc be able to handle multiple VM, one my svc instance:

vbox:vm1 –> svc for starting/stopping vm1,

vbox:vm2 –> svc for starting/stopping vm2,

An interested reader, Leo, gave the svc a try and felt into some problem when trying it with multiple VM: Only the svc instance of the VM that started first would stay running, while the other instances started after it would fall into maintenance. In the svc log file appeared:

[ Jul 23 03:38:39 Method "start" exited with status 0. ]
[ Jul 23 03:38:39 Stopping because all processes in service exited. ]
[ Jul 23 03:38:39 Executing stop method ("/export/vm/svc/method/vbox stop"). ]

The stangest thing is that although the svc instances were in maintenance mode, the corresponding VMs were actually running fine.

Here is an example, with 2 vbox instances: both vm1 and vm2 are running fine, but:

leo@os:~$ svcs -pv vbox
STATE          NSTATE        STIME    CTID   FMRI
online         -             13:51:42     99 svc:/site/xvm/vbox:vm1
               13:51:41      710 VBoxXPCOMIPCD
               13:51:41      713 VBoxSVC
               13:51:42      715 VBoxHeadless
               13:51:47      727 VBoxHeadless
maintenance    -             13:51:48      - svc:/site/xvm/vbox:vm2

So svc vbox:vm1 is online, while vbox:vm2 felt into maintenance. Notice that vbox:vm1 has 2 VBoxHeadless processes in its contract, while vbox:vm2 has no processes!

Actually, doing some tests together, we get to the point that the problem was with how Virtualbox works:

Upon starting the vbox:vm2 svc instance (at 13:51:47), there is already a VM running (vm1 started at 13:51:42). When the svc starts, VBoxManage startvm is actually “talking” to an already running Virtualbox process (either VBoxXPCOMIPCD or VBoxXVC, I suspect the first one, COMIPCD = Communication IPC Deamon?) and telling him to start the second VM. This “already running VB process” is the one that will fork (inside the contract of vbox:vm1, in this case ctid 99) and launch the VM, while the VBoxManage started by second svc will end. In this case, the VM is running from outside the contract of the svc, and when both the method (vbox) and the svcs itself return, the contract has no processes and then SMF detects the “failure” and try to restart it.

To help understand this, remember that the first version of my manifest was defining a contract model service, which means (see man page of svc.startd(1m)):

     A contract model service fails if any of the following  con-
     ditions occur:

         o    all processes in the service exit

         o    any processes in the service produce a core dump

         o    a process outside the service sends a service  pro-
              cess  a fatal signal (for example, an administrator
              terminates a service process with  the  pkill  com-
              mand)

The solution to the problem is changing the “startd/duration” property from “contract” (the default) to “transient”, as explain in the man page of svc.startd(1m):

     Additionally, svc.startd managed  services  can  define  the
     optional  properties  listed  below  in  the startd property
     group.

     startd/duration

         The duration property defines the  service's  model.  It
         can  be  set  to  transient,  child also known as "wait"
         model services, or contract (the default).
[...]
     Defining a service as transient means that  svc.startd  does
     not  track  processes  for that service. Thus, the potential
     faults  described  for  contract  model  services  are   not
     considered failures for transient services. A transient ser-
     vice only enters the maintenance state if one of the  method
     failure conditions occurs.

This is done by adding the following lines to the manifest, inside the <service> section and importing it again using svccfg(1M):

   <property_group name='startd' type='framework'>
      <propval name='duration' type='astring' value='transient' />
   </property_group>

Though many services are using the transient model, I confess that I was quite exited by the way SMF was able to handle process failures and restart the svc using the contract model, and having to change to the transient model is rather disappointing. However it’s a restriction due to how Virtualbox creates its own processes.

  • Service dependencies

Following the advice by Renaud Manus in this smf-discuss thread, I’ve added a dependency on milestone/multi-user-server,

  • Property value checking

Following another advice by Renaud Manus in the same smf-discuss thread, I’ve implemented some validity checking on the value of the “vm/stop_method” property. I do it inside the method, because I’m not (yet) familiar with the use of the Value Constraints from the new Template Extensions feature.

  • VM State checking

I’ve also implemented some checking on the state of the VM, so I only start it if it’s not already started, or stop it only if it’s running. It’s probably not the most exaustive checks that can be done, but it helps.

DOWNLOAD here the new versions of both files and feel free to use them under the MIT Licence:

If you are not confortable with modifying the files, please read first to my previous post to know what to modify.

Thanks go to: reader Leo and Renaud Manus from smf-discuss list at Opensolaris.org for there help, time and interest.

Updated: MIT Licence.

ShareThis / Compartelo

Start Virtualbox VMs at boot time in Opensolaris with SMF

Published on Tuesday July 21st, 2009 in Unix & Linux. 11 Comments Tags: , , .

At home I use Sun xVM Virtualbox in my Opensolaris server to run a Debian GNU/Linux virtual machine (my uTorrent server). As opposed to VMware Server, which let you start and stop your VMs at boot and shutdown time (under Linux), Virtualbox doesn’t provide any mechanism to do it. For Linux, there are lots of init scripts around the Internet (just ask google). For service startup, Opensolaris uses a new way introduced since Solaris 10, called SMF: Service Management Facility. This post is about creating a new service under Opensolaris’ SMF to handle my Virtualbox headless VMs‘ startup at boot time (and shutdown/standby when turning down the host).

Update 01/09/2009: If you are interested in trying and using this SMF svc yourself to start your Virtualbox VM, you may also read this post which updates both files.

To create a new SMF Service you need two things:

  • a service manifest: it’s a XML file which describes the service, and how SMF has to handle it: its dependencies (if any), its start and stop methods, its properties (if any), its working context (working directory, user…)
  • the method(s) file(s) script(s): that is the actual start/stop/refresh scripts (shell scripts for example).

Let’s fist start with the definition of the service, that is the SMF manifest file:

  • I name the service “site/xvm/vbox”:
<service
name='site/xvm/vbox'
type='service'
version='1.1'>
  • I define two dependencies (I could set more of course):
    • Network working (svc:/milestone/network:default)
    • Filesystem mounted (svc:/system/filesystem/local:default)
<dependency name='network' grouping='require_all' restart_on='none' type='service'>
   <service_fmri value='svc:/milestone/network:default' />
</dependency>

<dependency name='filesystem-local' grouping='require_all' restart_on='none' type='service'>
   <service_fmri value='svc:/system/filesystem/local:default' />
</dependency>
  • the declaration of the methods:
<exec_method
   type='method'
   name='start'
   exec='/export/home/adumont/svc/method/vbox start'
   timeout_seconds='60'
/>

<exec_method
   type='method'
   name='stop'
   exec='/export/home/adumont/svc/method/vbox stop'
timeout_seconds='60'
/>

You will probably need to change the path to the method from /export/home/adumont/svc to whatever path suit your instalation the most. Don’t forget to change it in the manifest, so the exec_method point to the an existing method script.

  • the definition of a service instance (in my case I create a service instance for every VM I want to handle with this service):
<instance name='debian' enabled='false'>
... 
</instance>
  • within the instance definition I define the working context:
    • working directory
    • processes credentials
   <method_context working_directory='/export/home/adumont/svc/var'>
      <method_credential user='adumont' group='staff' />
   </method_context>
  • and the instance properties: I set an instance property called vm/stop_method, which I’ll use from the stop method to know I the method should stop the VM (see later in this post):
   <property_group  name='vm' type='application'>
      <!-- stop_method is used to set how SMF will stop the VM:
      Possible values are: acpipowerbutton, savestate, acpisleepbutton, poweroff -->
      <propval name='stop_method' type='astring' value='savestate' />
   </property_group>

Here is the full xml manifest file (/export/home/adumont/svc/manifest/vbox-debian.xml):

<?xml version="1.0"?>
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">

<service_bundle type='manifest' name='vbox'>

<service
name='site/xvm/vbox'
type='service'
version='1.1'>

<!--   Wait for network interfaces to be initialized. -->
<dependency
name='network'
grouping='require_all'
restart_on='none'
type='service'>

<service_fmri value='svc:/milestone/network:default' />

</dependency>

<!--    Wait for all local filesystems to be mounted.    -->
<dependency
name='filesystem-local'
grouping='require_all'
restart_on='none'
type='service'>

<service_fmri value='svc:/system/filesystem/local:default' />

</dependency>

<exec_method
type='method'
name='start'
exec='/export/home/adumont/svc/method/vbox start'
timeout_seconds='60'
/>

<exec_method
type='method'
name='stop'
exec='/export/home/adumont/svc/method/vbox stop'
timeout_seconds='60'
/>

<instance name='debian' enabled='false'>

   <method_context working_directory='/export/home/adumont/svc/var'>
   <method_credential user='adumont' group='staff' />
   </method_context>

   <property_group  name='vm' type='application'>
   <!-- stop_method is used to set how SMF will stop the VM:
   Possible values are: acpipowerbutton, savestate, acpisleepbutton, poweroff -->
      <propval name='stop_method' type='astring' value='savestate' />
   </property_group>

</instance>

<stability value='Evolving' />

<template>
<common_name>
<loctext xml:lang='C'>Sun xVM Virtualbox</loctext>
</common_name>
<documentation>
<manpage title='Sun xVM Virtualbox' section='1' />
</documentation>
</template>

</service>

</service_bundle>

Now let’s see the method (/export/home/adumont/svc/method/vbox). The method defined in the manifest will be invoked upon service start and stop events (and also refresh, although I haven’t set any, so the “:true” method will be called. In my case I have written a single shell script that accept both start and stop arguments and will respond accordingly. The script is quite easy to understand, and is quite similar to old-style init startup scripts (you can have a look at many example of method scripts in /lib/svc/method/).

Let see in detail the script:

  • I first source the smf_include.sh file; this files is sourced by almost all the methods and define some usefull SMF variables and status constants that can be used within the method:
. /lib/svc/share/smf_include.sh
  • then, I check if the script is called from under the SMF framework or not:
# SMF_FMRI is the name of the target service. This allows multiple instances
# to use the same script.

if [ -z $SMF_FMRI ]; then
   echo "SMF framework variables are not initialized."
   exit $SMF_EXIT_ERR
fi
  • I then define a small function useful later to get the value of the vm/stop_method property of the instance:
getproparg() {
   val=`svcprop -p $1 $SMF_FMRI`
   [ -n "$val" ] && echo $val
}
  • later come the start and stop functions. Within the stop funtion, I first retrieve the value of the vm/stop_method property. If not found, I use a default value (“savestate”). Using this variable, I’ll be able to choose how the service stops the VM.
start_vm() {
   /usr/bin/VBoxManage startvm $1 --type vrdp
}

stop_vm() {
   # STOP_METHOD=acpipowerbutton|savestate
   STOP_METHOD="$( getproparg vm/stop_method )"

   [ -z "$STOP_METHOD" ] && STOP_METHOD="savestate"

   /usr/bin/VBoxManage controlvm $1 $STOP_METHOD
}
  • As the same method can be used to manage multiple instances (virtual machine) of the same vbox service, I have to retrive the name of the instance from the FMRI of the service:
INSTANCE=$( echo $SMF_FMRI | cut -d: -f3 )
  • finally I run the function that corresponds to the argument passed from the SMF framework upon calling the method (start/stop) and exit using an SMF friendly exit status (previously defined in smf_include.sh):
case $1 in
   start)
      start_vm $INSTANCE
      ;;
   stop)
      VM_STATE=$( vm_state $INSTANCE )

      if [ "x$VM_STATE" = "xrunning" ]
      then
         stop_vm $INSTANCE
      else
         echo "INFO: VM $INSTANCE in state $VM_STATE, I won't stop it."
      fi
      ;;
esac

if [ $? -ne 0 ]; then
   echo "ERROR: VM $INSTANCE failed to start/stop."
   exit $SMF_EXIT_ERR_FATAL
fi

exit $SMF_EXIT_OK

Here is the full method script (/export/home/adumont/svc/method/vbox, chmod 755):

#!/sbin/sh
#

. /lib/svc/share/smf_include.sh

# SMF_FMRI is the name of the target service. This allows multiple instances
# to use the same script.

if [ -z $SMF_FMRI ]; then
   echo "SMF framework variables are not initialized."
   exit $SMF_EXIT_ERR
fi

getproparg() {
   val=`svcprop -p $1 $SMF_FMRI`
   [ -n "$val" ] && echo $val
}

start_vm() {
   /usr/bin/VBoxManage startvm $1 --type vrdp
}

stop_vm() {
   # STOP_METHOD=acpipowerbutton|savestate
   STOP_METHOD="$( getproparg vm/stop_method )"

   [ -z "$STOP_METHOD" ] && STOP_METHOD="savestate"

   /usr/bin/VBoxManage controlvm $1 $STOP_METHOD
}

vm_state() {
   /usr/bin/VBoxManage showvminfo $1 --details --machinereadable |
      grep VMState\= | tr -s '"' ' ' | cut -d " " -f2

   if [ $? -ne 0 ]; then
      echo >&2 "ERROR: Failed to get VMState for VM $1"
      exit $SMF_EXIT_ERR_FATAL
   fi
}

INSTANCE=$( echo $SMF_FMRI | cut -d: -f3 )

case $1 in
   start)
      start_vm $INSTANCE
      ;;
   stop)
      VM_STATE=$( vm_state $INSTANCE )

      if [ "x$VM_STATE" = "xrunning" ]
      then
         stop_vm $INSTANCE
      else
         echo "INFO: VM $INSTANCE in state $VM_STATE, I won't stop it."
      fi
      ;;
esac

if [ $? -ne 0 ]; then
   echo "ERROR: VM $INSTANCE failed to start/stop."
   exit $SMF_EXIT_ERR_FATAL
fi

exit $SMF_EXIT_OK

Now that we’ve seen how to write the manifest and method files, let’s register our new SMF service, using first svccfg validate, and then svccfg import:

$ svccfg validate vbox-debian.xml
$ svccfg import vbox-debian.xml
$ svcs svc:/site/xvm/vbox:debian
STATE          STIME    FMRI
disabled       17:42:23 svc:/site/xvm/vbox:debian

The service has been created and is actually disabled. Let’s enable it:

$ svcadm -v enable -ts vbox:debian
svc:/site/xvm/vbox:debian temporarily enabled.

We can see the corresponding processes using ps:

$ ps -ef | grep VB
adumont 28515 28513   2 17:46:09 ?           0:03 /opt/VirtualBox/amd64/VBoxHeadless --comment debian --startvm 8b8505c3-87d5-49a
adumont 28513     1   0 17:46:08 ?           0:00 /opt/VirtualBox/amd64/VBoxSVC --automate
adumont 28510     1   0 17:46:08 ?           0:00 /opt/VirtualBox/amd64/VBoxXPCOMIPCD

Now let’s try to stop the VM:

$ svcadm -v disable -ts vbox:debian
$ ps -ef | grep VB
[no output]

it works! Let’s start it again, and now check the state again, using various command:

$ svcs -l vbox
fmri         svc:/site/xvm/vbox:debian
name         Sun xVM Virtualbox
enabled      true
state        online
next_state   none
state_time   Thu Jul 16 17:49:35 2009
logfile      /var/svc/log/site-xvm-vbox:debian.log
restarter    svc:/system/svc/restarter:default
contract_id  2159
dependency   require_all/none svc:/milestone/network:default (online)
dependency   require_all/none svc:/system/filesystem/local:default (online)
$ svcs -vx vbox
svc:/site/xvm/vbox:debian (Sun xVM Virtualbox)
State: online since Thu Jul 16 17:49:35 2009
See: man -M /usr/share/man -s 1 Sun xVM Virtualbox
See: /var/svc/log/site-xvm-vbox:debian.log
Impact: None.

Using the following command we can get the processes associated with the service:

$ svcs -p vbox
STATE          STIME    FMRI
online         17:49:35 svc:/site/xvm/vbox:debian
17:49:35    28568 VBoxXPCOMIPCD
17:49:35    28571 VBoxSVC
17:49:35    28573 VBoxHeadless

When starting a service, SMF start the associated processes in a “contract” (a group of processes). This is specially useful to handle service conditions like process failure and signals (core, kill,…). The contract information (ctid) can be seen using ctstat:

$ svcs -v vbox:debian
STATE          NSTATE        STIME      CTID   FMRI
online         -             17:49:35   2154   svc:/site/xvm/vbox:debian

$ ctstat -i 2154 -v
CTID    ZONEID  TYPE    STATE   HOLDER  EVENTS  QTIME   NTIME
2154    0       process owned   7       0       -       -
cookie:                0x20
informative event set: none
critical event set:    core signal hwerr empty
fatal event set:       none
parameter set:         inherit regent
member processes:      28568 28571 28573 
inherited contracts:   none
service fmri:          svc:/site/xvm/vbox:debian
service fmri ctid:     2154
creator:               svc.startd
aux:                   start

Now, let’s restart the VM:

$ svcadm -v restart vbox:debian
Action restart set for svc:/site/xvm/vbox:debian.

and see the corresponding log entries:

$ cat /var/svc/log/site-xvm-vbox:debian.log
...
[ Jul 16 18:05:44 Stopping because service restarting. ]
[ Jul 16 18:05:44 Executing stop method ("/export/home/adumont/svc/method/vbox stop"). ]
VirtualBox Command Line Management Interface Version 3.0.0
(C) 2005-2009 Sun Microsystems, Inc.
All rights reserved.

0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
[ Jul 16 18:05:46 Method "stop" exited with status 0. ]
[ Jul 16 18:05:52 Executing start method ("/export/home/adumont/svc/method/vbox start"). ]
VirtualBox Command Line Management Interface Version 3.0.0
(C) 2005-2009 Sun Microsystems, Inc.
All rights reserved.

Waiting for the remote session to open...
Remote session has been successfully opened.
[ Jul 16 18:05:53 Method "start" exited with status 0. ]

Finally, the way the SMF service will stop the VM can be changed by modifying the instance property vm/stop_method:

For example, if you want to orderly shutdown your VM upon service stop, you can set the property to acpipowerbutton:

$ svccfg -s vbox:debian setprop vm/stop_method=acpipowerbutton
$ svccfg -s vbox:debian refresh

if on the contrary you prefer to have the machine state frozen and saved to disk (by Virtualbox, not by the VM guest OS) use savestate:

$ svccfg -s vbox:debian setprop vm/stop_method=savestate
$ svccfg -s vbox:debian refresh

References:

site/xvm/vbox

ShareThis / Compartelo

Close
Powered by ShareThis