Salt's Reactor system gives Salt the ability to trigger actions in response to an event. It is a simple interface to watching Salt's event bus for event tags that match a given pattern and then running one or more commands in response.

该系统结合sls文件在master端去匹配event tags. 这些sls文件定义反应. 这意味着reactor系统有两个部分. 第一, reactor选项需要在master配置文件中进行设置. 这些reactor选项允许event tags与sls反应文件进行关联. 第二, 这些反应文件使用highdata(类似于state系统)去定义哪些反应应该执行.


理解reactor时需要首先基本理解event系统. event系统是一个本地的ZeroMQ PUB接口, 用于产生salt events. 这个event总线是一个开放的系统, 用于发送给Salt和其他系统发送关于操作的通告信息.

event系统产生event有一个严格的标准. 每一个event有一个 tag . event tags用于快速过滤events. 每一个event有一个数据结构附加在tag后. 这个数据结构是个字典, 包含关于本event的信息.

映射events到Reactor SLS文件

Reactor SLS文件和event tags在master配置文件中进行关联. 默认的配置文件是 /etc/salt/master或/etc/salt/master.d/reactor.conf.

2014.7.0 新版功能: Added Reactor support for salt:// file paths.

在master配置 'reactor:' 章节, 指定event tags列表来进行匹配, 每一个event tag有一组reactor SLS文件去运行.

reactor:                            # Master config section "reactor"

  - 'salt/minion/*/start':          # Match tag "salt/minion/*/start"
    - /srv/reactor/start.sls        # Things to do when a minion starts
    - /srv/reactor/monitor.sls      # Other things to do

  - 'salt/cloud/*/destroyed':       # Globs can be used to match tags
    - /srv/reactor/destroy/*.sls    # Globs can be used to match file names

  - 'myco/custom/event/tag':        # React to custom event tags
    - salt://reactor/mycustom.sls   # Put reactor files under file_roots

Reactor sls files are similar to state and pillar sls files. They are by default yaml + Jinja templates and are passed familiar context variables.

tagdata 变量的不同.

  • tag 变量对应的是产生的event的tag.

  • data 变量是event的数据字典.

这里是一个简单的reactor sls文件:

{% if data['id'] == 'mysql1' %}
    - tgt: mysql1
{% endif %}

这是一个简单的reactor文件使用Jinja去说明反应的产生. 如果event数据的 idmysql1 (换句话说, 如果该minion的名字是 mysql1)然后定义接下来的reaction. reactor系统中使用和state系统同样的数据结构和编译程序. 唯一不同的是数据使用的是salt command API和runner系统. 在本例中, state.highstate 函数命令被发送到 mysql1 minion上. 同样的, 一个runner是这样调用的:

{% if data['data']['orchestrate'] == 'refresh' %}
{% endif %}

This example will execute the state.orchestrate runner and initiate an orchestrate execution.

The Goal of Writing Reactor SLS Files

Reactor SLS files share the familiar syntax from Salt States but there are important differences. The goal of a Reactor file is to process a Salt event as quickly as possible and then to optionally start a new process in response.

  1. The Salt Reactor watches Salt's event bus for new events.
  2. The event tag is matched against the list of event tags under the reactor section in the Salt Master config.
  3. The SLS files for any matches are Rendered into a data structure that represents one or more function calls.
  4. That data structure is given to a pool of worker threads for execution.

Matching and rendering Reactor SLS files is done sequentially in a single process. Complex Jinja that calls out to slow Execution or Runner modules slows down the rendering and causes other reactions to pile up behind the current one. The worker pool is designed to handle complex and long-running processes such as Salt Orchestrate.

tl;dr: Rendering Reactor SLS files MUST be simple and quick. The new process started by the worker threads can be long-running.

Jinja Context

Reactor files only have access to a minimal Jinja context. grains and pillar are not available. The salt object is available for calling Runner and Execution modules but it should be used sparingly and only for quick tasks for the reasons mentioned above.

Advanced State System Capabilities

Reactor SLS files, by design, do not support Requisites, ordering, onlyif/unless conditionals and most other powerful constructs from Salt's State system.

Complex Master-side operations are best performed by Salt's Orchestrate system so using the Reactor to kick off an Orchestrate run is a very common pairing.

For example:

# /etc/salt/master.d/reactor.conf
# A custom event containing: {"foo": "Foo!", "bar: "bar*", "baz": "Baz!"}
  - myco/custom/event:
    - /srv/reactor/some_event.sls
# /srv/reactor/some_event.sls
    - mods: orch.do_complex_thing
    - pillar:
        event_tag: {{ tag }}
        event_data: {{ data | json() }}
# /srv/salt/orch/do_complex_thing.sls
{% set tag = salt.pillar.get('event_tag') %}
{% set data = salt.pillar.get('event_data') %}

# Pass data from the event to a custom runner function.
# The function expects a 'foo' argument.
    - name: custom_runner.custom_function
    - foo: {{ }}

# Wait for the runner to finish then send an execution to minions.
# Forward some data from the event down to the minion's state run.
    - tgt: {{ }}
    - sls:
      - do_thing_on_minion
    - pillar:
        baz: {{ data.baz }}
    - require:
      - salt: do_first_thing


To fire an event from a minion call event.send

salt-call event.send 'foo' '{orchestrate: refresh}'

After this is called, any reactor sls files matching event tag foo will execute with {{ data['data']['orchestrate'] }} equal to 'refresh'.

访问 salt.modules.event 获取更多信息.


The best way to see exactly what events are fired and what data is available in each event is to use the state.event runner.


salt-run state.event pretty=True


salt/job/20150213001905721678/new       {
    "_stamp": "2015-02-13T00:19:05.724583",
    "arg": [],
    "fun": "",
    "jid": "20150213001905721678",
    "minions": [
    "tgt": "*",
    "tgt_type": "glob",
    "user": "root"
salt/job/20150213001910749506/ret/jerry {
    "_stamp": "2015-02-13T00:19:11.136730",
    "cmd": "_return",
    "fun": "saltutil.find_job",
    "fun_args": [
    "id": "jerry",
    "jid": "20150213001910749506",
    "retcode": 0,
    "return": {},
    "success": true

Debug Reactor系统

最好的了解Reactor方法是将master运行在前台并启用debug log选项. 数据会包含master看到的event, 以及master回应该event的操作, 也包含SLS文件的渲染(或者在渲染SLS文件过程中出现的错误).

  1. 停止master.

  2. Start the master manually:

    salt-master -l debug
  3. Look for log entries in the form:

    [DEBUG   ] Gathering reactors for tag foo/bar
    [DEBUG   ] Compiling reactions for tag foo/bar
    [DEBUG   ] Rendered data from file: /path/to/the/reactor_file.sls:
    <... Rendered output appears here. ...>

    The rendered output is the result of the Jinja parsing and is a good way to view the result of referencing Jinja variables. If the result is empty then Jinja produced an empty result and the Reactor will ignore it.


I.e., when to use `arg` and `kwarg` and when to specify the function arguments directly.

While the reactor system uses the same basic data structure as the state system, the functions that will be called using that data structure are different functions than are called via Salt's state system. The Reactor can call Runner modules using the runner prefix, Wheel modules using the wheel prefix, and can also cause minions to run Execution modules using the local prefix.

在 2014.7.0 版更改: The cmd prefix was renamed to local for consistency with other parts of Salt. A backward-compatible alias was added for cmd.

The Reactor runs on the master and calls functions that exist on the master. In the case of Runner and Wheel functions the Reactor can just call those functions directly since they exist on the master and are run on the master.

In the case of functions that exist on minions and are run on minions, the Reactor still needs to call a function on the master in order to send the necessary data to the minion so the minion can execute that function.

The Reactor calls functions exposed in Salt's Python API documentation. and thus the structure of Reactor files very transparently reflects the function signatures of those functions.

Calling Execution modules on Minions

The Reactor sends commands down to minions in the exact same way Salt's CLI interface does. It calls a function locally on the master that sends the name of the function as well as a list of any arguments and a dictionary of any keyword arguments that the minion should use to execute that function.

Specifically, the Reactor calls the async version of this function. You can see that function has 'arg' and 'kwarg' parameters which are both values that are sent down to the minion.

Executing remote commands maps to the LocalClient interface which is used by the salt command. This interface more specifically maps to the cmd_async method inside of the LocalClient class. This means that the arguments passed are being passed to the cmd_async method, not the remote method. A field starts with local to use the LocalClient subsystem. The result is, to execute a remote command, a reactor formula would look like this:

    - tgt: '*'
    - arg:
      - rm -rf /tmp/*

arg 选项会携带一个参数列表, 就如同在命令行上指定. 因此以上的定义与运行如下salt命令作用相同:

salt '*' 'rm -rf /tmp/*'

使用 expr_form 参数来指定一个匹配器

    - tgt: 'os:Ubuntu'
    - expr_form: grain
    - arg:
      - rm -rf /tmp/*

    - tgt: 'G@roles:hbase_master'
    - expr_form: compound
    - arg:
      - rm -rf /tmp/*

Any other parameters in the LocalClient().cmd() method can be specified as well.

Calling Runner modules and Wheel modules

Calling Runner modules and Wheel modules from the Reactor uses a more direct syntax since the function is being executed locally instead of sending a command to a remote system to be executed there. There are no 'arg' or 'kwarg' parameters (unless the Runner function or Wheel function accepts a parameter with either of those names.)

For example:


If the the runner takes arguments then they must be specified as keyword arguments.

    - prof: centos_6
    - instances:
      - web11       # These VM names would be generated via Jinja in a
      - web12       # real-world example.

To determine the proper names for the arguments, check the documentation or source code for the runner function you wish to call.

Passing event data to Minions or Orchestrate as Pillar

从Reactor脚本的 state.highstatestate.sls 去传递Pillar数据的一个小技巧是这两个函数携带一个名为 pillar 的参数关键字.

接下来的例子将使用Salt的Reactor去监听在master端使用 ``salt-key``去接受一个新的minion时产生的event.


  - 'salt/key':
    - /srv/salt/haproxy/react_new_minion.sls

Reactor之后产生一个 state.sls 命令给HAProxy服务器, 通过内置的pillar将新minion的ID由event传递给state文件.


{% if data['act'] == 'accept' and data['id'].startswith('web') %}
    - tgt: 'haproxy*'
    - arg:
      - haproxy.refresh_pool
    - kwarg:
          new_minion: {{ data['id'] }}
{% endif %}


salt 'haproxy*' state.sls haproxy.refresh_pool 'pillar={new_minion: minionid}'

This works with Orchestrate files as well:

    - mods: some_orchestrate_file
    - pillar:
        stuff: things

Which is equivalent to the following command at the CLI:

salt-run state.orchestrate some_orchestrate_file pillar='{stuff: things}'

最终, 在state文件中的数据使用常规的Pillar查找语法. 接下来的例子将通过 :ref:Salt Mine <salt-mine>` 来获取webserver名字和IP地址. 如果召回来自于Reactor的state, 然后使用如上的自定义的Pillar, 一个新的minion会加入该pool中, 但会携带一个 disabled 标签, 因此HAProxy并不会导入流量到它上边.


{% set new_minion = salt['pillar.get']('new_minion') %}

listen web *:80
    balance source
    {% for server,ip in salt['mine.get']('web*', 'network.interfaces', ['eth0']).items() %}
    {% if server == new_minion %}
    server {{ server }} {{ ip }}:80 disabled
    {% else %}
    server {{ server }} {{ ip }}:80 check
    {% endif %}
    {% endfor %}

A Complete Example

在该例子中, 我们假设已有一个服务器组, 它会随机上线, 需要key自动接受. 我们并不想设置为所有的服务器均自动接受key. 因此本例中, 我们将嘉定所有id以'link'开头的主机的key会自动接受并执行state.highstate. On top of this, we're going to add that a host coming up that was replaced (meaning a new key) will also be accepted.

我们的master配置是相当简单的. 所以尝试认证的minion会匹配到名为 salt/authtag . 当minion key接受时, 我们会得到新的包含minion id可以用于匹配的的 tag.


  - 'salt/auth':
    - /srv/reactor/auth-pending.sls
  - 'salt/minion/ink*/start':
    - /srv/reactor/auth-complete.sls

在该sls文件中, 我们说如果key被回绝, 此时minion进程会死掉, 我们会删除master端的key并且告诉master去ssh到minion上重启该minion.

We also say that if the key is pending and the id starts with ink we will accept the key. A minion that is waiting on a pending key will retry authentication every ten seconds by default.


{# Ink server faild to authenticate -- remove accepted key #}
{% if not data['result'] and data['id'].startswith('ink') %}
    - match: {{ data['id'] }}
    - tgt: salt-master.domain.tld
    - arg:
      - ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no "{{ data['id'] }}" 'sleep 10 && /etc/init.d/salt-minion restart'
{% endif %}

{# Ink server is sending new key -- accept this key #}
{% if 'act' in data and data['act'] == 'pend' and data['id'].startswith('ink') %}
    - match: {{ data['id'] }}
{% endif %}

这里不需要使用if语法, 因为我们在master配置文件中已经限制本操作只针对刚刚加入的ink servers.


{# When an Ink server connects, run state.highstate. #}
    - tgt: {{ data['id'] }}
    - ret: smtp

The above will also return the highstate result data using the smtp_return returner (use virtualname like when using from the command line with --return). The returner needs to be configured on the minion for this to work. See salt.returners.smtp_return documentation for that.

Syncing Custom Types on Minion Start

Salt will sync all custom types (by running a saltutil.sync_all) on every highstate. However, there is a chicken-and-egg issue where, on the initial highstate, a minion will not yet have these custom types synced when the top file is first compiled. This can be worked around with a simple reactor which watches for minion_start events, which each minion fires when it first starts up and connects to the master.

On the master, create /srv/reactor/sync_grains.sls with the following contents:

    - tgt: {{ data['id'] }}

And in the master config file, add the following reactor configuration:

  - 'minion_start':
    - /srv/reactor/sync_grains.sls

This will cause the master to instruct each minion to sync its custom grains when it starts, making these grains available when the initial highstate is executed.

Other types can be synced by replacing local.saltutil.sync_grains with local.saltutil.sync_modules, local.saltutil.sync_all, or whatever else suits the intended use case.