LM "Actions"

I'd like to second this... We need this functionality (like nagios event handlers)

If ____ fails, restart service
If restarting service _____ times and _____ is still failing, THEN send an alert.

or worded example.. let's say dns resolution is failing.... rare but usually named needs to be restarted and life goes on... why do I need to be paged to do that at 2 in the morning when LM can do it for me ? :)/emoticons/smile@2x.png 2x" title=":)" width="20">

•  "dns resolution has failed"
• try to restart the service twice (user specified)
• If dns resolution has not resolved itself, then page that the service is down and that we have tried to restart the service twice without resolution.

Additional Note, it looks like this goes along with 

Hello,

We're currently looking to evaluate LogicMonitor as a potential replacement for Microsoft System Center Operations Manager (SCOM) and prior to SCOM being our enterprise monitoring tool we had IBM Tivoli Monitoring (ITM) in place within our organization.

So we come from well over 10+ years of being able to take corrective actions without the tools themselves in response to various alerts that are raised and based on our initial demo with the LogicMonitor team, we understand that's not a feature of the product as they don't want to be in config management business which I understand.

However, we can't be the only organization that has this issue so I'm curious how others have worked around this that would be willing to share their solutions. 

Here are some simple things we do today:

1. 1. Windows Service Restarts (we only alert in most cases if the corrective restart action fails)
2. 2. Linux Process Counts (we'll attempt to restart the process or execute some type of other scripted action)
3. 3. IIS Application Pool failures (we'll attempt using builtin Windows functionality to recycle and AppPool)

Appreciate the responses, thanks!

Hey @NBM,

Because our collector can act as a script-runner, we have some folks that have written scripts that look for those conditions, act on what they see, and then report back to LogicMonitor after taking one of those actions.

Check out @Mike Suding's blog post and DataSource for Windows Services as an example:

http://blog.mikesuding.com/index.php/2016/09/20/restart-a-service-alert-if-restart-fails/

Best,

Kerry

@Kerry DeVilbiss Thanks a ton!  We're working towards an official POC and appreciate the communities response on some of these initial questions we've had.  I'll definitely want to try this out to see how it functions once we get further along. 

I've created two more DataSources. One that restarts a Linux service and one for Windows that runs specified commands when it detects an alert.

Monitor a Linux Service and restart if needed:  http://blog.mikesuding.com/2018/12/27/service-restart-for-linux/

Generic run some PowerShell (Windows) commands or a script if alert happens:   http://blog.mikesuding.com/2018/12/03/automatic-action-triggered-by-an-alert/

This would be an INCREDIBLY USEFUL feature in my environment right now.  Mike's script looks great, but I can't seem to make it work.  I downloaded it, installed it, and added the monitored service, but it only ever shows a "4" and I get the errors below.  I thought it might be a permissions error or something so I jumped onto the collector using the same credentials and ran the commands in Powershell.  This worked fine.  Any ideas what might be wrong here?

Get-Service : Cannot find any service with service name 'hidserv'.
At C:\Program Files (x86)\LogicMonitor\Agent\tmp\scr1001001-Service_restart-Ser
vice_restart-Human_Interface_Device_Service.ps1:8 char:20
+ ... e_status = (Get-Service -Name $service -ComputerName $hostname).Statu ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (hidserv:String) [Get-Service],  
   ServiceCommandException
    + FullyQualifiedErrorId : NoServiceFoundForGivenName,Microsoft.PowerShell. 
   Commands.GetServiceCommand
 
get-service : Cannot find any service with service name 'hidserv'.
At C:\Program Files (x86)\LogicMonitor\Agent\tmp\scr1001001-Service_restart-Ser
vice_restart-Human_Interface_Device_Service.ps1:11 char:6
+ if ((get-service -name $service -ComputerName $hostname).Status -eq " ...
+      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (hidserv:String) [Get-Service],  
   ServiceCommandException
    + FullyQualifiedErrorId : NoServiceFoundForGivenName,Microsoft.PowerShell. 
   Commands.GetServiceCommand
 
Get-Service : Cannot find any service with service name 'hidserv'.
At C:\Program Files (x86)\LogicMonitor\Agent\tmp\scr1001001-Service_restart-Ser
vice_restart-Human_Interface_Device_Service.ps1:15 char:32
+ ... ervice -InputObject $(Get-Service -Computer $hostname -Name $service)
+                           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (hidserv:String) [Get-Service],  
   ServiceCommandException
    + FullyQualifiedErrorId : NoServiceFoundForGivenName,Microsoft.PowerShell. 
   Commands.GetServiceCommand
 
Start-Service : Cannot validate argument on parameter 'InputObject'. The 
argument is null or empty. Provide an argument that is not null or empty, and 
then try the command again.
At C:\Program Files (x86)\LogicMonitor\Agent\tmp\scr1001001-Service_restart-Ser
vice_restart-Human_Interface_Device_Service.ps1:15 char:30
+ ... ervice -InputObject $(Get-Service -Computer $hostname -Name $service)
+                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [Start-Service], ParameterBindi 
   ngValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.Power 
   Shell.Commands.StartServiceCommand
 
get-service : Cannot find any service with service name 'hidserv'.
At C:\Program Files (x86)\LogicMonitor\Agent\tmp\scr1001001-Service_restart-Ser
vice_restart-Human_Interface_Device_Service.ps1:21 char:8
+   if ((get-service -name $service -ComputerName $hostname).Status  -e ...
+        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (hidserv:String) [Get-Service],  
   ServiceCommandException
    + FullyQualifiedErrorId : NoServiceFoundForGivenName,Microsoft.PowerShell. 
   Commands.GetServiceCommand
 
Get-Service : Cannot find any service with service name 'hidserv'.
At C:\Program Files (x86)\LogicMonitor\Agent\tmp\scr1001001-Service_restart-Ser
vice_restart-Human_Interface_Device_Service.ps1:25 char:38
+ ... ervice -InputObject $(Get-Service -Computer $hostname -Name $service)
+                           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (hidserv:String) [Get-Service],  
   ServiceCommandException
    + FullyQualifiedErrorId : NoServiceFoundForGivenName,Microsoft.PowerShell. 
   Commands.GetServiceCommand
 
Start-Service : Cannot validate argument on parameter 'InputObject'. The 
argument is null or empty. Provide an argument that is not null or empty, and 
then try the command again.
At C:\Program Files (x86)\LogicMonitor\Agent\tmp\scr1001001-Service_restart-Ser
vice_restart-Human_Interface_Device_Service.ps1:25 char:36
+ ... ervice -InputObject $(Get-Service -Computer $hostname -Name $service)
+                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [Start-Service], ParameterBindi 
   ngValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.Power 
   Shell.Commands.StartServiceCommand
 
get-service : Cannot find any service with service name 'hidserv'.
At C:\Program Files (x86)\LogicMonitor\Agent\tmp\scr1001001-Service_restart-Ser
vice_restart-Human_Interface_Device_Service.ps1:31 char:14
+ ...        if ((get-service -name $service -ComputerName $hostname).Statu ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (hidserv:String) [Get-Service],  
   ServiceCommandException
    + FullyQualifiedErrorId : NoServiceFoundForGivenName,Microsoft.PowerShell. 
   Commands.GetServiceCommand