Nov 18 19:21:49 centos7 systemd: Starting SolarWinds Agent Service…
Nov 18 19:21:49 centos7 journal: SolarWinds Agent[28777]: SolarWinds Agent [1.8.0.551] Upgrade – deleting file [/opt/SolarWinds/Agent/bin/swiagent.upgrade]
Nov 18 19:21:49 centos7 systemd: Started SolarWinds Agent Service.
Nov 18 19:21:49 centos7 journal: SolarWinds Agent[28777]: SolarWinds Agent [1.8.0.551] Upgrade – deleting file [/opt/SolarWinds/Agent/bin/swiagent.upgrade]
Nov 18 19:21:51 centos7 journal: SolarWinds Agent[28777]: SolarWinds Agent (provisioning) [1.8.0.551] was started, built with OpenSSL version [OpenSSL 1.0.2k-fips 26 Jan 2017]
As we can see in the log events, the Agent is registered as a service and its installation files have been automatically deleted.
In security logs, we can also see how the installer creates a new local user in the system, which will be used to start services:
[root@centos7 log]# cat secure | grep swiagent
Nov 18 19:21:47 centos7 groupadd[28641]: group added to /etc/group: name=swiagent, GID=992
Nov 18 19:21:47 centos7 groupadd[28641]: group added to /etc/gshadow: name=swiagent
Nov 18 19:21:47 centos7 groupadd[28641]: new group: name=swiagent, GID=992
Nov 18 19:21:47 centos7 useradd[28646]: new user: name=swiagent, UID=995, GID=992, home=/opt/SolarWinds/Agent, shell=/sbin/nologin
[root@centos7 log]# pwd
/var/log
[root@centos7 log]#
[root@centos7 home]# cat /etc/passwd | grep SolarWinds
swiagent:x:995:992::/opt/SolarWinds/Agent:/sbin/nologin
[root@centos7 home]#
After installation, we can immediately see the agent process running in the background:
[root@centos7 bin]# ps xua | grep swiagent
swiagent 28777 0.3 1.9 1735676 19588 ? Ssl 19:21 0:03 /opt/SolarWinds/Agent/bin/swiagent /detach pidfile=/opt/SolarWinds/Agent/bin/swiagentd.pid
swiagent 28850 0.3 1.3 1018416 14024 ? Sl 19:22 0:02 /opt/SolarWinds/Agent/bin/Plugins/JobEngine/SolarWinds.Agent.JobEngine.Plugin ep=EWA4dd90f6d01924b579d7421f27782eb49 id=63c91981-0837-42d3-955a-dcc687f4bd8c.EWWP ipc=ipc_mq_v1
root 29695 0.0 0.1 119924 1172 pts/2 S+ 19:37 0:00 grep –color=auto swiagent
[root@centos7 bin]#
After successful installation of the agent, the network connection to the SolarWinds monitoring server should also be visible:
[root@centos7 log]# netstat –inet -n | grep 192.168.1.39
tcp 0 0 192.168.1.45:42636 192.168.1.39:17778 ESTABLISHED
tcp 0 0 192.168.1.45:42677 192.168.1.39:17778 ESTABLISHED
tcp 0 0 192.168.1.45:42676 192.168.1.39:17778 ESTABLISHED
The agent software uses the network port 17778 and SSL encryption, which is supported via the OpenSSL library.
It is very important to use the current version of OpenSSL, which has no known security vulnerability.
It is worth noting that we will not find commands initiated by the agent through the “history” command. This is due to the fact that all monitored statistics are obtained through the service, not the SSH connection. So we cannot use bash console history to view them.
However, we can view the agent’s logs and see what exactly is being initialized. The following example show log entries for the “Linux Disk Monitoring Perl” template used by the Server & Application Monitor module.
[root@centos7 Logs]# pwd
/opt/SolarWinds/Agent/bin/appdata/Logs
[root@centos7 Logs]#
[root@centos7 Logs]# cat APM_job_application.log
—-snip—-
17/11/18 19:52:28.573 PID: 30317 TID: 140015132571456 [INFO] job_application – Starting poll of component 561 (Available space on / partition (MB))
17/11/18 19:52:28.580 PID: 30317 TID: 140015132571456 [DEBUG] scriptrunner – Initial identity: uid=0, gid=992, euid=995, egid=992
17/11/18 19:52:28.580 PID: 30317 TID: 140015132571456 [DEBUG] scriptrunner – User identity before: uid=0, gid=992, euid=995, egid=992
17/11/18 19:52:28.580 PID: 30317 TID: 140015132571456 [DEBUG] scriptrunner – Creating temp script file: /tmp/APM_3QI_oS
17/11/18 19:52:28.581 PID: 30317 TID: 140015132571456 [INFO] scriptrunner – Execution of command (subprocess method): perl /tmp/APM_3QI_oS
17/11/18 19:52:28.604 PID: 30317 TID: 140015132571456 [DEBUG] scriptrunner – Script output:
Message: Available space on / partition in MB : 36851
Statistic: 36851
17/11/18 19:52:28.604 PID: 30317 TID: 140015132571456 [DEBUG] scriptrunner – Script exit code: 0
17/11/18 19:52:28.605 PID: 30317 TID: 140015132571456 [DEBUG] scriptrunner – File /tmp/APM_3QI_oS deleted
17/11/18 19:52:28.605 PID: 30317 TID: 140015132571456 [DEBUG] scriptrunner – User identity after: uid=0, gid=992, euid=995, egid=992
17/11/18 19:52:28.608 PID: 30317 TID: 140015132571456 [DEBUG] job_application – job_application.py exiting
In the example above, we can clearly see the agent’s actions. The agent creates a temporary Perl script, executes it (the script checks disk statistics) and then deletes it.
The action is performed regularly in predefined cycles, every 300 seconds by default.
