DFSR Replication Backlog
XKJNGZ Uses Powershell to make WMI queries to get the current backlog file count for each outbound DFSR partner on each DFSR share. These queries can be expensive if the backlog is large, so the polling interval is set to 10 minutes. If there is no backlog, the script finishes quickly. No default alerting is set, but I would recommend adding a threshold to be notified of potential replicationissues. NOTE* - The collector must be able to reach both DFSR partners and will use the same credentials to make the queries for both.
Windows System Event Log "message" details not accurate
We are using the defaultWindows System Event Log event source and having those errors route through a Teams integration. When tested fromWindows System Event Log event source the Event Logging displays the entire “message” detailing the eventID reason etc etc. When looking in the Alerts section of the GUI it also shows the entire “Message” section with details. However when the alert shows up in Teams its dumbed down and useless. We get the following. Message: error - HOSTNAME Windows System Event Log The Teams integration is setup identically to the Event Source Alert message as seen below. Anyone know why ##Message## is getting overwritten with useless info instead of the actual message details from the Event? Host: ##HOST## Eventsource: ##EVENTSOURCE## Windows Event ID: ##EVENTCODE## Message: ##MESSAGE## Detected on: ##START##When an anomaly isn't an anomaly what could i do?
What can i do when anomaly detection wont work ( something that is seen on a regular basis, and dynamic threshold also wont help where it is within range? For example a drive on a server gets filled with data ( drive is normally cleared down on a daily basis ) but when someone decides to upload a larger than expected amount the drive hasn't been cleared or with other uploads throughout the day there isn't enough space. You are happy if the drive is above 80% during the night because if it hasn't cleared it can be dealt with in the morning ( no need to get anyone out of bed ) but if there is a rapid spike ( more than 2.5% growth in used space in a 30min period ) then they need an alert to get out of bed and fix / make enough room for the data. A possible solution is a datasourcethat will alert if the drive is over the 80% but only with that rapid growth. DataSource calls the api for the last 30min worth of data and calculates the growth rate. The below is the code for a C drive but the drive letter can be changed easily in the code below, same with the 2.5% and the 80% values, they could also be parameterised for different ranges on different devices. <# Use TLS 1.2 #> [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 <# account info #> $accessId = '##apiaccessid.key##' $accessKey = '##apiaccesskey.key##' $company = '##company##' $deviceId = "##system.deviceId##" <# request details #> $httpVerb = 'GET' $resourcePath = "/device/devices/$deviceId/devicedatasources" $queryParams = '?filter=dataSourceName:"WinVolumeUsage-"' <# Construct URL #> $url = 'https://' + $company + '.logicmonitor.com/santaba/rest' + $resourcePath + $queryParams <# Get current time in milliseconds #> $epoch = [Math]::Round((New-TimeSpan -start (Get-Date -Date "1/1/1970") -end (Get-Date).ToUniversalTime()).TotalMilliseconds) <# Concatenate Request Details #> $requestVars = $httpVerb + $epoch + $data + $resourcePath <# Construct Signature #> $hmac = New-Object System.Security.Cryptography.HMACSHA256 $hmac.Key = [Text.Encoding]::UTF8.GetBytes($accessKey) $signatureBytes = $hmac.ComputeHash([Text.Encoding]::UTF8.GetBytes($requestVars)) $signatureHex = [System.BitConverter]::ToString($signatureBytes) -replace '-' $signature = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($signatureHex.ToLower())) <# Construct Headers #> $auth = 'LMv1 ' + $accessId + ':' + $signature + ':' + $epoch $headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]" $headers.Add("Authorization",$auth) $headers.Add("Content-Type",'application/json') $headers.Add("X-Version","3") <# Make Request #> $response = Invoke-RestMethod -Uri $url -Method $httpVerb -Header $headers <# Get Device DataSource ID #> $deviceDataSourceId = $response.items.id <# request details #> $httpVerb = 'GET' $resourcePath = "/device/devices/$deviceId/devicedatasources/$deviceDataSourceId/data" $queryParams = '' <# Construct URL #> $url = 'https://' + $company + '.logicmonitor.com/santaba/rest' + $resourcePath + $queryParams <# Get current time in milliseconds #> $epoch = [Math]::Round((New-TimeSpan -start (Get-Date -Date "1/1/1970") -end (Get-Date).ToUniversalTime()).TotalMilliseconds) <# Concatenate Request Details #> $requestVars = $httpVerb + $epoch + $data + $resourcePath <# Construct Signature #> $hmac = New-Object System.Security.Cryptography.HMACSHA256 $hmac.Key = [Text.Encoding]::UTF8.GetBytes($accessKey) $signatureBytes = $hmac.ComputeHash([Text.Encoding]::UTF8.GetBytes($requestVars)) $signatureHex = [System.BitConverter]::ToString($signatureBytes) -replace '-' $signature = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($signatureHex.ToLower())) <# Construct Headers #> $auth = 'LMv1 ' + $accessId + ':' + $signature + ':' + $epoch $headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]" $headers.Add("Authorization",$auth) $headers.Add("Content-Type",'application/json') <# Make Request #> $response = Invoke-RestMethod -Uri $url -Method $httpVerb -Header $headers <# Print status and body of response #> $status = $response.status $body = $response.data | ConvertTo-Json -Depth 5 function Select-Nth { param([int]$N) $Input | Select-Object -First $N | Select-Object -Last 1 } $array1 = @($response.data.instances.'WinVolumeUsage-C:\'.values) $first = $array1[0] | Select-Nth 3 $last = $array1[19] |Select-Nth 3 $growth = $first - $last if (($growth -gt 2.5) -and ($first -ge 80)){ return 1 }else { return 2 } Hope this gives you some ideas to develop alerting further😁
Datasource to monitor Windows Services/Processes automatically?
Hello, We recently cloned 2 Logic Monitor out of the box datasources (name ->WinService- & WinProcessStats-) in order to enable the 'Active Discovery' feature on those. We did this becausewe've the need to discover services/processesautomatically, since we don't have an 'exact list' of which services/processes we should monitor (due to the amount of clients [+100] & the different services/solutions across them) After enabling this it works fine & does what we expect (discovers all the services/processes running in each box),we further added some filters in the active discovery for the servicesin order to exclude common 'noisy' services & grab only the ones set to automatically start with the system. Our problem arrives when these 2specific datasourcestartto impact the collector performance (due to the huge amount of wmi.queries), it starts to reflect on a huge consumption of CPU(putting thaton almost 100% usage all the time) & that further leads to the decrease of the collector performance & data collection (resulting in request timeouts & full WMI queues). We also thought on creating 2 datasources(services/processes) for each client (with filters to grab critical/wanted processes/services for the client in question) but that's a nightmare(specially when you've clients installing applications without any notice & expecting us to automatically grab & monitor those). Example of 1 of our scenarios (1of our clients): - Collector is a Windows VM (VMWare)&has 8GB of RAM with4 allocated virtual processors (host processor is a Intel Xeon E5-2698v3 @ 2.30Ghz) - Currently, it monitors 78 Windows servers (not including the collector) & those 2datasourceare creating 12 700 instances (4513 - services | 8187 - processes) - examples below This results in approx. 15 requests per second This results in approx. 45 requests per second According to the collector capacity document (ref. Medium Collector) we are below the limits (forWMI), however, those 2 datasourceare contributing A LOT to make the queues full. We're finding errors in a regular basis- example below To sum thisup, we were seeking for another 'way' of doing the same thing without consuming so much resources on the collector end (due to the amount of simultaneousWMI queries). Not sure if that's possible though. Did anyone had this need in the past & was able to come up with a differentsolution (not so resource exhaustive)? We're struggling here mainly because we come from a non-agent less solution (which didn't facedthis problem due to the individual agentdistributed load - per device). Appreciate the help in advance! Thanks,