Manage plugins in python with importlib

Yoann Dupont (Almanach)


developer meetup, Febuary 19th 2019

Plugin management?


  • purpose: extend app with new functionalities or implement an interface (user code)
  • plugin: external code to be integrated into a running program
  • plugin manager: a software facility used by a running program to discover and load code, often containing hooks called by the host application (Mahmoud Hashemi at PyCon 2017)

importlib — overview

  • python source code for the import statement
  • have a built-in import system for python
  • give devs a tool for finding/loading code more easily
  • documentation is big: only a tiny part here!
  • more than one way to do it: differences in rationale and operation

importlib — examples (from PyPA)

naming conventions

import importlib
import pkgutil

myapp_plugins = {
    name: importlib.import_module(name)
    for finder, name, ispkg in pkgutil.iter_modules()
    if name.startswith('myapp_')
}

namespace packages

import importlib
import pkgutil
import myapp.plugins

def iter_namespace(ns_pkg):
    return pkgutil.iter_modules(
        ns_pkg.__path__,
        ns_pkg.__name__ + "."
    )

myapp_plugins = {
    name: importlib.import_module(name)
    for finder, name, ispkg
    in iter_namespace(myapp.plugins)
}

package metadata (setuptools)

setup(
    ...
    entry_points={'myapp.plugins': 'a = myapp_plugin_a'},
    ...
)

import pkg_resources

myapp_plugins = {
    entry_point.name: entry_point.load()
    for entry_point
    in pkg_resources.iter_entry_points('myapp.plugins')
}

a plugin folder (educational purpose)

import importlib

def load_plugins(plugins_path):
    modules = []
    for file_path in plugins_path.glob("*.py"):
        module_name = str(file_path.stem)
        spec = importlib.util.spec_from_file_location(
            module_name,
            str(file_path)
        )
        module = importlib.util.module_from_spec(spec)
        spec.loader.exec_module(module)
        modules.append((module_name, module))
    return tuple(modules)

myapp_plugins = {
    module_name: module
    for module_name, module
    in load_plugins(myapp.USER_PLUGINS_PATH)
}

Ressources and related content