加載heat resource的代碼分析

Heat的核心是stack，stack又是由各種各樣的資源組成的，heat除了自定義的大量資源外，還允許用戶自定義自己需要的資源。

heat資源加載流程

我們先從heat-engine的啓動腳本來看:

heat/cmd/engine.py
if __name__ == '__main__':  
  
    cfg.CONF(project='heat', prog='heat-engine')  
    logging.setup('heat')  
    messaging.setup()  
  
    mgr = None  
    try:  
	#加載heat的模板類  
        mgr = template._get_template_extension_manager()  
    except template.TemplatePluginNotRegistered as ex:  
        LOG.critical(_LC("%s"), ex)  
    if not mgr or not mgr.names():  
        sys.exit("ERROR: No template format plugins registered")  
  
    from heat.engine import service as engine  
  
    srv = engine.EngineService(cfg.CONF.host, rpc_api.ENGINE_TOPIC)  

上面爲heat-egine進程啓動的代碼，在創建EngineService的時候，這裏面有個resources初始化的步驟,

這裏應該就是完成資源加載的關鍵，如下面的代碼所示

heat/engine/service.py EngineService.__init__():

def __init__(self, host, topic, manager=None):  
        super(EngineService, self).__init__()  
        resources.initialise() #對資源進行初始化，分析代碼可以除了初始化資源，此處還會初始化clients. 
        self.host = host  
        self.topic = topic  

首先是有個全局變量_environment，由於是初次加載，所以會執行clients.initialise()

heat/engine/resources/__init__.py
def initialise():  
    global _environment  
    if _environment is not None:  
        return  
  
    clients.initialise()  
  
    global_env = environment.Environment({}, user_env=False)  
    _load_global_environment(global_env)  
    _environment = global_env  

clients其實就是heat與各個openstack組件通信的關鍵，類似的，這裏採用了全局_mgr來管理clients的加載，

初次加載的時候由於全局變量_mgr爲None，所以採用stevedore加載heat需要用到的各種clients，

_mgr = None  
  
  
def has_client(name):  
    return _mgr and name in _mgr.names()  
  
heat/engine/clients/__init__.py 
def initialise():  
    global _mgr  
    if _mgr:  
        return  
  
    _mgr = extension.ExtensionManager(  
        namespace='heat.clients',  
        invoke_on_load=False,  
        verify_requirements=False) 

這裏的clients爲entrypoints中定義的名稱空間爲heat.clients中的各種client,具體可以查看heat的api-paste.ini文件。

加載完這些clients後，會定義一個Environment對象：

heat/engine/environment.py
def __init__(self, env=None, user_env=True):  
        """Create an Environment from a dict of varying format. 
        1) old-school flat parameters 
        2) or newer {resource_registry: bla, parameters: foo} 
 
        :param env: the json environment 
        :param user_env: boolean, if false then we manage python resources too. 
        """  
        if env is None:  
            env = {}  
        if user_env:  
            from heat.engine import resources  
            global_registry = resources.global_env().registry  
        else:  
            global_registry = None  
  
        self.registry = ResourceRegistry(global_registry)  
        self.registry.load(env.get(env_fmt.RESOURCE_REGISTRY, {}))  
  （部分代碼有刪減）

後面我們加載資源的關鍵了

self.registry = ResourceRegistry(global_registry)  def _load_global_environment(env):  
    _load_global_resources(env)  
    environment.read_global_environment(env)  
  
  
def _load_global_resources(env):  
    _register_constraints(env, _get_mapping('heat.constraints'))  
    _register_stack_lifecycle_plugins(  
        env,  
        _get_mapping('heat.stack_lifecycle_plugins'))  
  
    manager = plugin_manager.PluginManager(__name__)  
    # Sometimes resources should not be available for registration in Heat due  
    # to unsatisfied dependencies. We look first for the function  
    # 'available_resource_mapping', which should return the filtered resources.  
    # If it is not found, we look for the legacy 'resource_mapping'.  
    resource_mapping = plugin_manager.PluginMapping(['available_resource',  
                                                     'resource'])  
    constraint_mapping = plugin_manager.PluginMapping('constraint')  
  
    _register_resources(env, resource_mapping.load_all(manager))  
  
    _register_constraints(env, constraint_mapping.load_all(manager))

在_load_global_resources方法裏面，首先是往env裏面註冊constraints和stack_lifecycle_plugins，

註冊的原理很簡單，也是用stevedore進行插件的加載，需要加載的項在api-paste.init中的entry_points裏面可以看到。

然後通過下面的方法，將這些插件註冊到前面定義的global_env對象中，接下面就是resources的加載的，

這裏定義了2個插件相關的類，PluginManager以及PluginMapping，首先看下PluginManager

heat/engine/plugin_manager.py
class PluginManager(object):  
    '''''A class for managing plugin modules.'''  
  
    def __init__(self, *extra_packages):  
        '''''Initialise the Heat Engine plugin package, and any others. 
 
        The heat.engine.plugins package is always created, if it does not 
        exist, from the plugin directories specified in the config file, and 
        searched for modules. In addition, any extra packages specified are 
        also searched for modules. e.g. 
 
        >>> PluginManager('heat.engine.resources') 
 
        will load all modules in the heat.engine.resources package as well as 
        any user-supplied plugin modules. 
 
        '''  
        def packages():  
            for package_name in extra_packages:  
                yield sys.modules[package_name]  
  
            cfg.CONF.import_opt('plugin_dirs', 'heat.common.config')  
            yield plugin_loader.create_subpackage(cfg.CONF.plugin_dirs,  
                                                  'heat.engine')  
  
        def modules():  
            pkg_modules = itertools.imap(plugin_loader.load_modules,  
                                         packages())  
            return itertools.chain.from_iterable(pkg_modules)  
  
        self.modules = list(modules())  

其屬性modules主要是列出了資源所在的各個模塊，這裏默認加載的'heat.engine.resources'包下面的各個模塊，

此外，heat還允許我們自己配置加載資源插件的路徑，但是默認來說，heat自身只會加載'heat.engine.resources'下面的模塊，

所以我們擴展資源插件的時候，可以選擇放在這個目錄heat/engine/resources/。

接下來定義了2個PluginMapping，分別針對resources和constraints，注意到這裏傳入的參數，resources傳入的是

available_resource和resource，後面我們會看到爲什麼要傳這個.

resource_mapping= plugin_manager.PluginMapping(['available_resource',  
                                                    'resource']) 
constraint_mapping = plugin_manager.PluginMapping('constraint')

接着會執行下面這段代碼

_register_resources(env, resource_mapping.load_all(manager))  
def _register_resources(env, type_pairs):  
    for res_name, res_class in type_pairs:  
        env.register_class(res_name, res_class) 

那type_pairs裏面到底是啥呢，我們看下相關的方法，首先是resource_mapping.load_all(manager)的返回，

可以看到這裏返回了個迭代器,迭代器中的內容又是一個迭代器，每個迭代器的內容又來源於itertools.imap(function,self.modules)，

也就是將load_from_module方法作用於之前加載起來的在resources下面的各個資源模塊。

 def load_all(self, plugin_manager):  
        '''''Iterate over the mappings from all modules in the plugin manager. 
 
        Mappings are returned as a list of (key, value) tuples. 
        '''  
        mod_dicts = plugin_manager.map_to_modules(self.load_from_module)  
        return itertools.chain.from_iterable(six.iteritems(d) for d  
def map_to_modules(self, function):  
        '''''Iterate over the results of calling a function on every module.'''  
        return itertools.imap(function, self.modules) 

我們來看下PluginMapping中的load_from_module這個方法，可以看到，之前傳的available_resource和resource參數就起作用了，

該方法會從這個模塊嘗試去取available_resource_mapping和resource_mapping，如果available_resource_mapping或resource_mapping是函數，就會取resouce_mapping方法的內容

heat/engine/plugin_manager.py

def load_from_module(self, module):  
        '''''Return the mapping specified in the given module. 
 
        If no such mapping is specified, an empty dictionary is returned. 
        '''  
        for mapping_name in self.names:  
            mapping_func = getattr(module, mapping_name, None)  
            if callable(mapping_func):  
                fmt_data = {'mapping_name': mapping_name, 'module': module}  
                try:  
                    mapping_dict = mapping_func(*self.args, **self.kwargs)  
     (部分代碼有刪減）

我們以heat自帶的autoscaling模塊爲例，它的返回如下

def resource_mapping():  
    return {  
        'AWS::AutoScaling::AutoScalingGroup': AutoScalingGroup,  
        'OS::Heat::InstanceGroup': InstanceGroup,  
        'OS::Heat::AutoScalingGroup': AutoScalingResourceGroup,  
    }  

所以接着上面的代碼看，這裏的res_name和res_class就是resource_mapping或者available_mapping返回的字

典的內容，然後往之前的env對象中註冊該資源.

def _register_resources(env, type_pairs):  
    for res_name, res_class in type_pairs:  
        env.register_class(res_name, res_class)  
def register_class(self, resource_type, resource_class):  
        ri = ResourceInfo(self, [resource_type], resource_class)  
        self._register_info([resource_type], ri)

這裏首先根據我們傳入的資源創建一個合適的資源類，然後註冊到env中，直至把module中的資源加載完成。

加載constraints的過程也和resource類似，這裏不繼續展開。

當env加載完各種插件後，回到最之前的代碼會把這個global_env對象賦值給全局變量_environment，

def initialise():  
    global _environment  
    if _environment is not None:  
        return  
  
    clients.initialise()  
  
    global_env = environment.Environment({}, user_env=False)  
    _load_global_environment(global_env)  
    _environment = global_env 

添加自定義的資源到heat中

現在我們在resource目錄下自定義一個自己的插件，按照上面的分析，可以這麼寫，讓heat能夠正確加載我們的插件

from heat.engine import resource  
  
  
class MyResource(resource.Resource):  	
        def handle_create(self):
		pass
	def handle_update(self):
		pass
	def handle_delete(self):
		pass
      
  
def available_resource_mapping():  
    return {'OS::Openstack::MyRSC': MyResource}  

#我們在加載資源插件之後加一句打印代碼
def _load_global_resources(env):  
    _register_constraints(env, _get_mapping('heat.constraints'))  
    _register_stack_lifecycle_plugins(  
        env,  
        _get_mapping('heat.stack_lifecycle_plugins'))  
  
    manager = plugin_manager.PluginManager(__name__)  
    # Sometimes resources should not be available for registration in Heat due  
    # to unsatisfied dependencies. We look first for the function  
    # 'available_resource_mapping', which should return the filtered resources.  
    # If it is not found, we look for the legacy 'resource_mapping'.  
    resource_mapping = plugin_manager.PluginMapping(['available_resource',  
                                                     'resource'])  
    constraint_mapping = plugin_manager.PluginMapping('constraint')  
  
    _register_resources(env, resource_mapping.load_all(manager))  
  
    _register_constraints(env, constraint_mapping.load_all(manager))  
    print 'OS::Openstack::MyRSC' in env.registry._registry  

可以發現打印爲True，代表我們已經成功加載我們自定義的插件了.