SPM Development Guide

This document discusses developing additional code for SPM.

SPM-Specific Loader Modules

SPM was designed to behave like traditional package managers, which apply files to the filesystem and store package metadata in a local database. However, because modern infrastructures often extend beyond those use cases, certain parts of SPM have been broken out into their own set of modules.

Each function that accepts arguments has a set of required and optional arguments. Take note that SPM will pass all arguments in, and therefore each function must accept each of those arguments. However, arguments that are marked as required are crucial to SPM's core functionality, while arguments that are marked as optional are provided as a benefit to the module, if it needs to use them.

Package Database

By default, the package database is stored using the sqlite3 module. This module was chosen because support for SQLite3 is built into Python itself.

Modules for managing the package database are stored in the salt/spm/pkgdb/ directory. A number of functions must exist to support database management.

init()

Get a database connection, and initialize the package database if necessary.

This function accepts no arguments. If a database is used which supports a connection object, then that connection object is returned. For instance, the sqlite3 module returns a connect() object from the sqlite3 library:

conn = sqlite3.connect(__opts__['spm_db'], isolation_level=None)
...
return conn

SPM itself will not use this connection object; it will be passed in as-is to the other functions in the module. Therefore, when you set up this object, make sure to do so in a way that is easily usable throughout the module.

info()

Return information for a package. This generally consists of the information that is stored in the FORMULA file in the package.

The arguments that are passed in, in order, are package (required) and conn (optional).

package is the name of the package, as specified in the FORMULA. conn is the connection object returned from init().

list_files()

Return a list of files for an installed package. Only the filename should be returned, and no other information.

The arguments that are passed in, in order, are package (required) and conn (optional).

package is the name of the package, as specified in the FORMULA. conn is the connection object returned from init().

register_pkg()

Register a package in the package database. Nothing is expected to be returned from this function.

The arguments that are passed in, in order, are name (required), formula_def (required), and conn (optional).

name is the name of the package, as specified in the FORMULA. formula_def is the contents of the FORMULA file, as a dict. conn is the connection object returned from init().

register_file()

Register a file in the package database. Nothing is expected to be returned from this function.

The arguments that are passed in are name (required), member (required), path (required), digest (optional), and conn (optional).

name is the name of the package.

member is a tarfile object for the package file. It is included, because it contains most of the information for the file.

path is the location of the file on the local filesystem.

digest is the SHA1 checksum of the file.

conn is the connection object returned from init().

unregister_pkg()

Unregister a package from the package database. This usually only involves removing the package's record from the database. Nothing is expected to be returned from this function.

The arguments that are passed in, in order, are name (required) and conn (optional).

name is the name of the package, as specified in the FORMULA. conn is the connection object returned from init().

unregister_file()

Unregister a package from the package database. This usually only involves removing the package's record from the database. Nothing is expected to be returned from this function.

The arguments that are passed in, in order, are name (required), pkg (optional) and conn (optional).

name is the path of the file, as it was installed on the filesystem.

pkg is the name of the package that the file belongs to.

conn is the connection object returned from init().

db_exists()

Check to see whether the package database already exists. This is the path to the package database file. This function will return True or False.

The only argument that is expected is db_, which is the package database file.

Package Files

By default, package files are installed using the local module. This module applies files to the local filesystem, on the machine that the package is installed on.

Modules for managing the package database are stored in the salt/spm/pkgfiles/ directory. A number of functions must exist to support file management.

init()

Initialize the installation location for the package files. Normally these will be directory paths, but other external destinations such as databases can be used. For this reason, this function will return a connection object, which can be a database object. However, in the default local module, this object is a dict containing the paths. This object will be passed into all other functions.

Three directories are used for the destinations: formula_path, pillar_path, and reactor_path.

formula_path is the location of most of the files that will be installed. The default is specific to the operating system, but is normally /srv/salt/.

pillar_path is the location that the pillar.example file will be installed to. The default is specific to the operating system, but is normally /srv/pillar/.

reactor_path is the location that reactor files will be installed to. The default is specific to the operating system, but is normally /srv/reactor/.

check_existing()

Check the filesystem for existing files. All files for the package will be checked, and if any are existing, then this function will normally state that SPM will refuse to install the package.

This function returns a list of the files that exist on the system.

The arguments that are passed into this function are, in order: package (required), pkg_files (required), formula_def (formula_def), and conn (optional).

package is the name of the package that is to be installed.

pkg_files is a list of the files to be checked.

formula_def is a copy of the information that is stored in the FORMULA file.

conn is the file connection object.

install_file()

Install a single file to the destination (normally on the filesystem). Nothing is expected to be returned from this function.

This function returns the final location that the file was installed to.

The arguments that are passed into this function are, in order, package (required), formula_tar (required), member (required), formula_def (required), and conn (optional).

package is the name of the package that is to be installed.

formula_tar is the tarfile object for the package. This is passed in so that the function can call formula_tar.extract() for the file.

member is the tarfile object which represents the individual file. This may be modified as necessary, before being passed into formula_tar.extract().

formula_def is a copy of the information from the FORMULA file.

conn is the file connection object.

remove_file()

Remove a single file from file system. Normally this will be little more than an os.remove(). Nothing is expected to be returned from this function.

The arguments that are passed into this function are, in order, path (required) and conn (optional).

path is the absolute path to the file to be removed.

conn is the file connection object.

hash_file()

Returns the hexdigest hash value of a file.

The arguments that are passed into this function are, in order, path (required), hashobj (required), and conn (optional).

path is the absolute path to the file.

hashobj is a reference to hashlib.sha1(), which is used to pull the hexdigest() for the file.

conn is the file connection object.

This function will not generally be more complex than:

def hash_file(path, hashobj, conn=None):
    with salt.utils.fopen(path, 'r') as f:
        hashobj.update(f.read())
        return hashobj.hexdigest()

path_exists()

Check to see whether the file already exists on the filesystem. Returns True or False.

This function expects a path argument, which is the absolute path to the file to be checked.

path_isdir()

Check to see whether the path specified is a directory. Returns True or False.

This function expects a path argument, which is the absolute path to be checked.