6.2. Writing New Application KernelsΒΆ

While the current set of available application kernels might provide a good set of tools to start, sooner or later you will probably want to use a tool for which no application Kernel exsits. This section describes how you can add your custom kernels.

We have two files, user_script.py which contains the user application which uses our custom kernel, new_kernel.py which contains the definition of the custom kernel. You can download them from the following links:

Let’s first take a look at new_kernel.py.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
from radical.ensemblemd.kernel_plugins.kernel_base import KernelBase

# ------------------------------------------------------------------------------
#
_KERNEL_INFO = {
    "name":         "sleep",                  # Mandatory
    "description":  "sleeping kernel",        # Optional
    "arguments":   {                          # Mandatory
        "--interval=": {
            "mandatory": True,                # Mandatory argument? True or False
            "description": "Number of seconds to do nothing."
    	    },
        },
    "machine_configs":                        # Use a dictionary with keys as
        {                                     # resource names and values specific
            "local.localhost":                # to the resource
            {
                "environment" : None,         # dict or None, can be used to set env variables
                "pre_exec"    : None,         # list or None, can be used to load modules
                "executable"  : ["/bin/sleep"],        # specify the executable to be used
                "uses_mpi"    : False         # mpi-enabled? True or False
            },
        }
}

# ------------------------------------------------------------------------------
#
class MyUserDefinedKernel(KernelBase):

    def __init__(self):

        super(MyUserDefinedKernel, self).__init__(_KERNEL_INFO)
     	"""Le constructor."""
        		
    # --------------------------------------------------------------------------
    #
    @staticmethod
    def get_name():
        return _KERNEL_INFO["name"]
        

    def _bind_to_resource(self, resource_key):
        """This function binds the Kernel to a specific resource defined in
        "resource_key".
        """        
        arguments  = ['{0}'.format(self.get_arg("--interval="))]

        self._executable  = _KERNEL_INFO["machine_configs"][resource_key]["executable"]
        self._arguments   = arguments
        self._environment = _KERNEL_INFO["machine_configs"][resource_key]["environment"]
        self._uses_mpi    = _KERNEL_INFO["machine_configs"][resource_key]["uses_mpi"]
        self._pre_exec    = _KERNEL_INFO["machine_configs"][resource_key]["pre_exec"]

# ------------------------------------------------------------------------------

Lines 5-24 contain information about the kernel to be defined. “name” and “arguments” keys are mandatory. The “arguments” key needs to specify the arguments the kernel expects. You can specify whether the individual arguments are mandatory or not. “machine_configs” is not mandatory, but creating a dictionary with resource labels as keys and values which are resource specific lets use the same kernel to be used on different machines.

In lines 28-52, we define a user defined class (of “KernelBase” type) with 3 mandatory functions. First the constructor, self-explanatory. Second, a static method that is used by EnsembleMD to differentiate kernels. Third, _bind_to_resource which is the function that (as the name suggests) binds the kernel with its resource specific values, during execution. In lines 48, 50-52, you can see how the “machine_configs” dictionary approach is helps us across different resources. The values in lines 48-52 are the definitions of the kernel.

Now, let’s take a look at user_script.py

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
from radical.ensemblemd import Kernel
from radical.ensemblemd import Pipeline
from radical.ensemblemd import EnsemblemdError
from radical.ensemblemd import SingleClusterEnvironment

#Used to register user defined kernels
from radical.ensemblemd.engine import get_engine

#Import our new kernel
from new_kernel import MyUserDefinedKernel

# Register the user-defined kernel with Ensemble MD Toolkit.
get_engine().add_kernel_plugin(MyUserDefinedKernel)


#Now carry on with your application as usual !
class Sleep(Pipeline):

    def __init__(self,steps,instances):
        Pipeline.__init__(self, steps,instaces)

    def step_1(self, instance):
        """This step sleeps for 60 seconds."""

        k = Kernel(name="sleep")
        k.arguments = ["--interval=10"]
        return k


# ------------------------------------------------------------------------------
#
if __name__ == "__main__":

    try:
        # Create a new static execution context with one resource and a fixed
        # number of cores and runtime.
        cluster = SingleClusterEnvironment(
                resource="local.localhost",
                cores=1,
                walltime=15,
        	   username=None,
        	    project=None
        	)

        # Allocate the resources.
        cluster.allocate()

        # Set the 'instances' of the pipeline to 16. This means that 16 instances
        # of each pipeline step are executed.
        #
        # Execution of the 16 pipeline instances can happen concurrently or
        # sequentially, depending on the resources (cores) available in the
        # SingleClusterEnvironment.
        sleep = Sleep(steps=1,instances=16)

        cluster.run(sleep)

        cluster.deallocate()

    except EnsemblemdError, er:

        print "Ensemble MD Toolkit Error: {0}".format(str(er))
        raise # Just raise the execption again to get the backtrace

There are 3 important lines in this script. In line 7, we import the get_engine function in order to register our new kernel. In line 10, we import our new kernel and in line 13, we register our kernel. THAT’S IT. We can continue with the application as in the previous examples.