YAML语言

YAML是一种直观的能够被电脑识别的数据序列化格式，是一个可读性高并且容易被人类阅读，容易和脚本语言交互，用来表达资料序列的编程语言。

它类似于标准通用标记语言的子集XML的数据描述语言，语法比XML简单很多。

YAML语言的格式如下：

house:family:name: Doeparents:- John- Janechildren:- Paul- Mark- Simoneaddress:number: 34street: Main Streetcity: Nowheretownzipcode: 12345

YAML的基本规则：

使用缩进来表示层级关系，每层2个空格，禁止使用TAB键
当冒号不是处于最后时，冒号后面必须有一个空格
用 - 表示列表，- 的后面必须有一个空格
用 # 表示注释
YAML配置文件要放到SaltStack让我们放的位置，可以在SaltStack的 Master 配置文件中查找file_roots即可看到。

[root@master salt]# pwd
/etc/salt[root@master salt]# vim master
681 file_roots:682   base:683     - /srv/salt/base684   test:685     - /srv/salt/test686   dev:687     - /srv/salt/dev688   prod:689     - /srv/salt/prod[root@master ~]# mkdir -p /srv/salt/{base,test,dev,prod}[root@master base]# systemctl restart salt-master.service
[root@master base]# systemctl restart salt-minion.service

需要注意：
base是默认的位置，如果file_roots只有一个，则base是必备的且必须叫base，不能改名

使用SaltStack配置一个apache实例

在Master上部署sls配置文件并执行

[root@master base]# pwd
/srv/salt/base[root@master base]# mkdir -p web/apache
[root@master apache]# pwd
/srv/salt/base/web/apache[root@master apache]# touch apache.sls  //生成一个状态描述文件
[root@master apache]# vim apache.sls
apache-install:pkg.installed:- name: httpdapache-service:service.running:- name: httpd- enable: True// YAML 配置文件中顶格写的被称作ID，必须全局唯一，不能重复
// SaltStack 读 YAML 配置文件时是从上往下读，所以要把先执行的写在前面在执行状态文件前先用test.ping测试需要执行状态文件的主机是否能正常通信，然后再执行状态文件。
[root@master salt]# salt 'minion01' test.ping
minion01:True//执行状态描述文件
[root@master ~]# salt 'minion01' state.sls web.apache.apache saltenv=base
minion01:
----------ID: apache-installFunction: pkg.installedName: httpdResult: TrueComment: The following packages were installed/updated: httpdStarted: 08:42:04.287654Duration: 18111.181 msChanges:   ----------apr:----------new:1.6.3-9.el8old:apr-util:----------new:1.6.1-6.el8old:apr-util-bdb:----------new:1.6.1-6.el8old:apr-util-openssl:----------new:1.6.1-6.el8old:httpd:----------new:2.4.37-21.module+el8.2.0+5008+cca404a3old:httpd-filesystem:----------new:2.4.37-21.module+el8.2.0+5008+cca404a3old:httpd-tools:----------new:2.4.37-21.module+el8.2.0+5008+cca404a3old:mailcap:----------new:2.1.48-3.el8old:mod_http2:----------new:1.11.3-3.module+el8.2.0+4377+dc421495old:redhat-logos-httpd:----------new:81.1-1.el8old:
----------ID: apache-serviceFunction: service.runningName: httpdResult: TrueComment: Service httpd has been enabled, and is runningStarted: 08:42:22.458329Duration: 758.569 msChanges:   ----------httpd:TrueSummary for minion01
------------
Succeeded: 2 (changed=2)
Failed:    0
------------
Total states run:     2
Total run time:  18.870 s在minion上检查apache的状态
[root@minion01 salt]# systemctl status httpd
● httpd.service - The Apache HTTP ServerLoaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: d>Active: active (running) since Tue 2021-11-03 08:42:22 CST; 40min agoDocs: man:httpd.service(8)Main PID: 111233 (httpd)

使用SaltStack在minion02上配置nginx实例

在Master上部署sls配置文件并执行

[root@master web]# pwd
/srv/salt/base/web
[root@master web]# mkdir nginx[root@master nginx]# pwd
/srv/salt/base/web/nginx
[root@master nginx]# vim nginx.sls
nginx-install:pkg.installed:- name: nginx
nginx-service:service.running:- name: nginx- enable: true[root@master base]# salt 'minion02' state.sls web.nginx.nginx saltenv=base
minion02:
----------ID: nginx-installFunction: pkg.installedName: nginxResult: TrueComment: The following packages were installed/updated: nginxStarted: 09:36:09.485277Duration: 11087.069 msChanges:   ----------centos-indexhtml:----------new:7-9.el7.centosold:gperftools-libs:----------new:2.6.1-1.el7old:nginx:----------new:1:1.20.1-9.el7old:nginx-filesystem:----------new:1:1.20.1-9.el7old:openssl11-libs:----------new:1:1.1.1g-3.el7old:
----------ID: nginx-serviceFunction: service.runningName: nginxResult: TrueComment: Service nginx has been enabled, and is runningStarted: 09:36:20.603264Duration: 192.469 msChanges:   ----------nginx:TrueSummary for minion02
------------
Succeeded: 2 (changed=2)
Failed:    0
------------
Total states run:     2
Total run time:  11.280 s//在minion02上检查
[root@minion02 ~]# systemctl status nginx.service 
● nginx.service - The nginx HTTP and reverse proxy serverLoaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; vendor preset: disabled)Active: active (running) since 二 2021-11-03 09:36:20 CST; 1min 19s agoMain PID: 20747 (nginx)

top file介绍

直接通过命令执行sls文件时够自动化吗？答案是否定的，因为我们还要告诉某台主机要执行某个任务，自动化应该是我们让它干活时，它自己就知道哪台主机要干什么活，但是直接通过命令执行sls文件并不能达到这个目的，为了解决这个问题，top file 应运而生。

top file就是一个入口，top file的文件名可通过在 Master的配置文件中搜索top.sls找出，且此文件必须在 base 环境中，默认情况下此文件必须叫top.sls。

top file的作用就是告诉对应的主机要干什么活，比如让web服务器启动web服务，让数据库服务器安装mysql等等。
案例

[root@master base]# pwd
/srv/salt/base[root@master base]# touch top.sls
[root@master base]# vim top.sls
base:   //要执行状态文件的环境'minion01':  //要执行状态文件的目标- web.apache.apache  //要执行的状态文件//停掉minion01上的httpd服务，进行测试
[root@minion01 salt]# systemctl stop httpd.service[root@master base]# salt 'minion01' state.highstate  //使用高级状态来执行
minion01:
----------ID: apache-installFunction: pkg.installedName: httpdResult: TrueComment: All specified packages are already installedStarted: 10:31:02.863779Duration: 713.364 msChanges:   
----------ID: apache-serviceFunction: service.runningName: httpdResult: TrueComment: Service httpd is already enabled, and is runningStarted: 10:31:03.580843Duration: 362.207 msChanges:   ----------httpd:TrueSummary for minion01
------------
Succeeded: 2 (changed=1)
Failed:    0
------------
Total states run:     2
Total run time:   1.076 s//再来查看minion的httpd状态
[root@minion01 ~]# systemctl status httpd
● httpd.service - The Apache HTTP ServerLoaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor p>Active: active (running) since Tue 2021-11-03 10:31:03 CST; 1h 6min agoDocs: man:httpd.service(8)Main PID: 201195 (httpd)

使用top file来管理minion02上的nginx

[root@master base]# pwd
/srv/salt/base
[root@master base]# vim top.sls
base:'minion02':- web.nginx.nginx[root@minion02 ~]# systemctl stop nginx.service[root@master base]# salt 'minion02' state.highstate
minion02:
----------ID: nginx-installFunction: pkg.installedName: nginxResult: TrueComment: All specified packages are already installedStarted: 10:46:06.330211Duration: 631.054 msChanges:   
----------ID: nginx-serviceFunction: service.runningName: nginxResult: True
[root@master base]# salt 'minion02' state.highstate
minion02:
----------ID: nginx-installFunction: pkg.installedName: nginxResult: TrueComment: All specified packages are already installedStarted: 18:46:38.797394Duration: 626.71 msChanges:   
----------ID: nginx-serviceFunction: service.runningName: nginxResult: TrueComment: Service nginx is already enabled, and is runningStarted: 10:46:39.426389Duration: 115.327 msChanges:   ----------nginx:TrueSummary for minion02
------------
Succeeded: 2 (changed=1)
Failed:    0
------------
Total states run:     2
Total run time: 742.037 ms[root@minion02 ~]# systemctl status nginx.service 
● nginx.service - The nginx HTTP and reverse proxy serverLoaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; vendor preset: disabled)Active: active (running) since 二 2021-11-03 10:46:39 CST; 22s ago

同时管理minion01和02

[root@minion01 ~]# systemctl status httpd
[root@minion02 ~]# systemctl stop nginx.service
[root@master base]# salt '*' state.highstate
[root@master base]# salt '*' state.highstate
minion02:
----------ID: nginx-installFunction: pkg.installedName: nginxResult: TrueComment: All specified packages are already installedStarted: 10:49:51.691119Duration: 828.843 msChanges:   
----------ID: nginx-serviceFunction: service.runningName: nginxResult: TrueComment: Service nginx is already enabled, and is runningStarted: 10:49:52.523394Duration: 179.223 msChanges:   ----------nginx:TrueSummary for minion02
------------
Succeeded: 2 (changed=1)
Failed:    0
------------
Total states run:     2
Total run time:   1.008 s
minion01:
----------ID: apache-installFunction: pkg.installedName: httpdResult: TrueComment: All specified packages are already installedStarted: 10:49:50.300689Duration: 625.583 msChanges:   
----------ID: apache-serviceFunction: service.runningName: httpdResult: TrueComment: Service httpd is already enabled, and is runningStarted: 10:49:50.928605Duration: 315.648 msChanges:   ----------httpd:TrueSummary for minion01
------------
Succeeded: 2 (changed=1)
Failed:    0
------------
Total states run:     2
Total run time: 941.231 ms[root@minion01 ~]# systemctl status httpd
● httpd.service - The Apache HTTP ServerLoaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: d>Active: active (running) since Tue 2021-11-03 10:49:51 CST; 33s ago[root@minion02 ~]# systemctl status nginx.service 
● nginx.service - The nginx HTTP and reverse proxy serverLoaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; vendor preset: disabled)Active: active (running) since 二 2021-11-03 10:49:52 CST; 49s ago

注意： 若top file里面的目标是用表示的，要注意的是，top file里面的表示的是所有要执行状态的目标，而salt ‘’ state.highstate里面的表示通知所有机器干活，而是否要干活则是由top file来指定的

高级状态highstate的使用

管理SaltStack时一般最常用的管理操作就是执行高级状态。

[root@master salt]# salt '*' state.highstate  //在生产环境中不要使用此操作

注意： 上面让所有人执行高级状态，但实际工作中，一般不会这么用，工作当中一般都是通知某台或某些台目标主机来执行高级状态，具体是否执行则是由top file来决定的。

若在执行高级状态时加上参数test=True，则它会告诉我们它将会做什么，但是它不会真的去执行这个操作。

停掉minion上的httpd服务
[root@minion01 ~]# systemctl stop httpd在master上执行高级状态的测试
[root@master salt]# salt 'minion01' state.highstate test=true
minion01:
----------ID: apache-installFunction: pkg.installedName: httpdResult: TrueComment: All specified packages are already installedStarted: 09:54:34.099749Duration: 540.741 msChanges:   
----------ID: apache-serviceFunction: service.runningName: httpdResult: NoneComment: Service httpd is set to startStarted: 09:54:34.642736Duration: 60.081 msChanges:   Summary for minion01
------------
Succeeded: 2 (unchanged=1)
Failed:    0
------------
Total states run:     2
Total run time: 600.822 ms[root@minion01 ~]# systemctl status httpd
● httpd.service - The Apache HTTP ServerLoaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: dis>Active: inactive (dead) since Tue 2021-11-03 09:52:46 CST; 3min 15s agoDocs: man:httpd.service(8)//可以看出httpd并没有，说明高级命令并没有执行

SaltStack数据系统

SaltStack有两大数据系统，分别是：

Grains
Pillar
Grains与Pillar的区别

数据系统	存储位置	类型	采集方式	应用场景
Grains	minion	静态	minion启动时采集；可通过刷新避免重启minion服务	信息查询 在命令行下进行目标匹配 在top file中进行目标匹配 在模板中进行目标匹配
Pillar	master	动态	指定，实时生效	目标匹配 敏感数据配置

SaltStack组件之Grains

Grains是saltstack的一个组件，其存放着minion启动时收集到的信息。

Grains是saltstack组件中非常重要的组件之一，因为我们在做配置部署的过程中会经常使用它，Grains是saltstack记录minion的一些静态信息的组件。可简单理解为Grains记录着每台minion一些常用属性，比如CPU、内存、磁盘、网络信息等。我们可以通过grains.items查看某台minion的所有Grains信息。

Grains的功能：
收集资产信息

Grains应用场景：
信息查询
在命令行下进行目标匹配
在top file中进行目标匹配
在模板中进行目标匹配
模板中的目标匹配请看
https://docs.saltproject.io/en/latest/topics/pillar/

信息查询实列：

列出minion01所有grains的key和value值
[root@master base]# salt 'minion01' grains.items
minion01:----------biosreleasedate:  //bios的时间02/27/2020biosversion:   //bios的版本6.00cpu_flags:   //cpu相关的属性- fpu- vme- de- pse- tsc- msr- pae- mce- cx8- apic- sep- mtrr- pge- mca- cmov- pat- pse36- clflush- mmx- fxsr- sse- sse2- ht- syscall- nx- mmxext- fxsr_opt- pdpe1gb- rdtscp- lm- constant_tsc- rep_good- nopl- tsc_reliable- nonstop_tsc- cpuid- extd_apicid- pni- pclmulqdq- ssse3- fma- cx16- sse4_1- sse4_2- x2apic- movbe- popcnt- aes- xsave- avx- f16c- rdrand- hypervisor- lahf_lm- cmp_legacy- svm- extapic- cr8_legacy- abm- sse4a- misalignsse- 3dnowprefetch- osvw- perfctr_core- ssbd- ibpb- vmmcall- fsgsbase- bmi1- avx2- smep- bmi2- rdseed- adx- smap- clflushopt- sha_ni- xsaveopt- xsavec- xsaves- clzero- arat- npt- svm_lock- nrip_save- vmcb_clean- flushbyasid- decodeassists- overflow_recov- succorcpu_model:    //cpu的具体型号AMD Ryzen 5 3500U with Radeon Vega Mobile Gfxcpuarch:    //cpu架构x86_64cwd:/disks:- sr0dns:----------domain:ip4_nameservers:- 192.168.71.2ip6_nameservers:nameservers:- 192.168.71.2options:search:sortlist:domain:efi:Falseefi-secure-boot:Falsefqdn:minion01fqdn_ip4:    //ip地址- 192.168.71.140fqdn_ip6:- fe80::20c:29ff:fef4:b4a9fqdns:- minion01gid:0gpus:|_----------model:SVGA II Adaptervendor:vmwaregroupname:roothost:     //主机名minion01hwaddr_interfaces:----------ens160:00:0c:29:f4:b4:a9lo:00:00:00:00:00:00id:      //minion的IDminion01init:systemdip4_gw:192.168.71.2ip4_interfaces:----------ens160:- 192.168.71.140lo:- 127.0.0.1ip6_gw:Falseip6_interfaces:----------ens160:- fe80::20c:29ff:fef4:b4a9lo:- ::1ip_gw:Trueip_interfaces:----------ens160:- 192.168.71.140- fe80::20c:29ff:fef4:b4a9lo:- 127.0.0.1- ::1ipv4:- 127.0.0.1- 192.168.71.140ipv6:- ::1- fe80::20c:29ff:fef4:b4a9kernel:Linuxkernelparams:|_- BOOT_IMAGE- (hd0,msdos1)/vmlinuz-4.18.0-193.el8.x86_64|_- root- /dev/mapper/rhel-root|_- ro- None|_- crashkernel- auto|_- resume- /dev/mapper/rhel-swap|_- rd.lvm.lv- rhel/root|_- rd.lvm.lv- rhel/swap|_- rhgb- None|_- quiet- Nonekernelrelease:4.18.0-193.el8.x86_64kernelversion:#1 SMP Fri Mar 27 21:35:58 UTC 2020locale_info:----------defaultencoding:UTF-8defaultlanguage:zh_CNdetectedencoding:UTF-8timezone:CSTlocalhost:minion01lsb_distrib_codename:Red Hat Enterprise Linux 8.2 (Ootpa)lsb_distrib_id:Red Hat Enterprise Linuxlsb_distrib_release:8.2lvm:----------rhel:- home- root- swapmachine_id:b8032364c46f4fadaefd58290347f0ebmanufacturer:VMware, Inc.master:192.168.71.134mdadm:mem_total:1800nodename:minion01num_cpus:2num_gpus:1os:RedHatos_family:RedHatosarch:x86_64oscodename:Ootpaosfinger:Red Hat Enterprise Linux-8osfullname:Red Hat Enterprise Linuxosmajorrelease:8osrelease:8.2osrelease_info:- 8- 2path:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/binpid:49239productname:VMware Virtual Platformps:ps -efHwwpythonexecutable:/usr/bin/python3.6pythonpath:- /usr/bin- /usr/lib64/python36.zip- /usr/lib64/python3.6- /usr/lib64/python3.6/lib-dynload- /usr/lib64/python3.6/site-packages- /usr/lib/python3.6/site-packagespythonversion:- 3- 6- 8- final- 0saltpath:/usr/lib/python3.6/site-packages/saltsaltversion:3004saltversioninfo:- 3004selinux:----------enabled:Falseenforced:Disabledserialnumber:VMware-56 4d af 2f 62 90 b5 d0-b0 1b e2 f8 e8 f4 b4 a9server_id:1293882994shell:/bin/shssds:- nvme0n1swap_total:2091systemd:----------features:+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacyversion:239systempath:- /usr/local/sbin- /usr/local/bin- /usr/sbin- /usr/bintransactional:Falseuid:0username:rootuuid:2faf4d56-9062-d0b5-b01b-e2f8e8f4b4a9virtual:VMwarezfs_feature_flags:Falsezfs_support:Falsezmqversion:4.3.4

只查询所有的grains的key

[root@master base]# salt 'minion01' grains.ls
minion01:- biosreleasedate- biosversion- cpu_flags- cpu_model- cpuarch- cwd- disks- dns- domain- efi- efi-secure-boot- fqdn- fqdn_ip4- fqdn_ip6- fqdns- gid- gpus- groupname- host- hwaddr_interfaces- id- init- ip4_gw- ip4_interfaces- ip6_gw- ip6_interfaces- ip_gw- ip_interfaces- ipv4- ipv6- kernel- kernelparams- kernelrelease- kernelversion- locale_info- localhost- lsb_distrib_codename- lsb_distrib_id- lsb_distrib_release- lvm- machine_id- manufacturer- master- mdadm- mem_total- nodename- num_cpus- num_gpus- os- os_family- osarch- oscodename- osfinger- osfullname- osmajorrelease- osrelease- osrelease_info- path- pid- productname- ps- pythonexecutable- pythonpath- pythonversion- saltpath- saltversion- saltversioninfo- selinux- serialnumber- server_id- shell- ssds- swap_total- systemd- systempath- transactional- uid- username- uuid- virtual- zfs_feature_flags- zfs_support- zmqversion

查询某个key的值，比如想获取ip地址

[root@master base]# salt 'minion01' grains.get fqdn_ip4
minion01:- 192.168.71.140[root@master base]# salt 'minion02' grains.get fqdn_ip4
minion02:- 192.168.71.135//获取某个主机mac地址
[root@master base]# salt 'minion01' grains.get hwaddr_interfaces
minion01:----------ens160:00:0c:29:f4:b4:a9lo:00:00:00:00:00:00[root@master base]# salt '*' grains.get ip4_interfaces
minion01:----------ens160:- 192.168.71.140lo:- 127.0.0.1
minion02:----------ens33:- 192.168.71.135lo:- 127.0.0.1[root@master base]# salt 'minion01' grains.get hwaddr_interfaces:ens160
minion01:00:0c:29:f4:b4:a9

目标匹配实例：
用Grains来匹配minion：

在所有centos系统中执行命令
[root@master base]# salt -G 'os:CentOS' test.ping
minion02:True//在top file里面使用Grains：
[root@master base]# pwd
/srv/salt/base[root@master base]# vim top.sls
base:'os:CentOS':  //表示所有的centos系统都会执行要达到的状态- match: grain- web.nginx.nginx[root@master base]# salt -G 'os:CentOS' state.highstate
[root@minion02 ~]# systemctl stop nginx.service
[root@minion02 ~]# systemctl status nginx.service 
● nginx.service - The nginx HTTP and reverse proxy serverLoaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; vendor preset: disabled)Active: active (running) since 二 2021-11-03 11:14:54 CST; 9s ago

自定义Grains的两种方法：

minion配置文件，在配置文件中搜索grains
在/etc/salt下生成一个grains文件，在此文件中定义(推荐方式）

记得重启服务，因为minion只会读取重启之前的的内容，因为我们在minion开启状态下写的所以需要重启minion服务使其读取生效
[root@minion01 ~]# vim /etc/salt/grains
system: linux
version: redhat[root@minion01 ~]# systemctl restart salt-minion.service[root@master base]# salt 'minion01' grains.items
swap_total:2091system:linuxversion:redhatvirtual:若不想重启也可以使用下面的方法
[root@minion01 ~]# vim /etc/salt/grains
age: 18[root@master base]# salt 'minion01' saltutil.sync_grains
minion01:
[root@master base]# salt 'minion01' grains.get age
minion01:18

SaltStack组件之Pillar

Pillar也是SaltStack组件中非常重要的组件之一，是数据管理中心，经常配置states在大规模的配置管理工作中使用它。Pillar在SaltStack中主要的作用就是存储和定义配置管理中需要的一些数据，比如软件版本号、用户名密码等信息，它的定义存储格式与Grains类似，都是YAML格式。

在Master配置文件中有一段Pillar settings选项专门定义Pillar相关的一些参数：

#pillar_roots:
#  base:
#    - /srv/pillar

默认Base环境下Pillar的工作目录在/srv/pillar目录下。若你想定义多个环境不同的Pillar工作目录，只需要修改此处配置文件即可。

Pillar的特点：
可以给指定的minion定义它需要的数据
只有指定的人才能看到定义的数据
在master配置文件里设置

//查看pillar的信息
[root@master base]# salt '*' pillar.items
minion02:----------
minion01:----------

默认pillar是没有任何信息的，如果想查看信息，需要在 master 配置文件上把 pillar_opts的注释取消，并将其值设为 True

[root@master ~]# vim /etc/salt/master
pillar_roots:base:- /srv/pillar/bashpillar_opts: True  //取消注释，并将Fales改为True，这里是为了展示效果才开启，建议保持关闭状态。[root@master ~]# systemctl restart salt-master.service[root@master ~]# salt 'minion01' pillar.items
minion01:----------master:----------__cli:salt-master__role:masterallow_minion_key_revoke:True

指定top file入口文件

[root@master base]# pwd
/srv/pillar/base
[root@master base]# vim top.sls
base:   //指定环境'minion*':   //指定目标- apache   //引用apache.sls[root@master base]# salt '*' pillar.items
minion02:----------apache:test
minion01:----------apache:httpd

在salt下修改apache的状态文件，引用pillar的数据

[root@master base]# cat /srv/salt/base/web/apache/apache.sls
apache-install:pkg.installed:- name: {{ pillar['apache'] }}apache-service:service.running:- name: {{ pillar['apache']}}- enable: True

执行高级状态文件

[root@master base]# salt '*' state.highstate 
minion02:
----------ID: statesFunction: no.NoneResult: FalseComment: No Top file or master_tops data matches found. Please see master log for details.Changes:   Summary for minion02
------------
Succeeded: 0
Failed:    1
------------
Total states run:     1
Total run time:   0.000 ms
minion01:
----------ID: apache-installFunction: pkg.installedName: httpdResult: TrueComment: All specified packages are already installedStarted: 11:45:39.585376Duration: 721.042 msChanges:   
----------ID: apache-serviceFunction: service.runningName: httpdResult: TrueComment: The service httpd is already runningStarted: 11:45:40.310194Duration: 115.002 msChanges:   Summary for minion01
------------
Succeeded: 2
Failed:    0
------------
Total states run:     2
Total run time: 836.044 ms
ERROR: Minions returned with non-zero exit code