Forum Discussion

domenicbetters's avatar
2 years ago

Is there a way to set custom property values from your groovy script in your datasource?

Hey there.  I’m relatively new to logic monitor and I received a request from a client.  

The request is as follows.  

They wanted us to be able to provide not only blocked processes alerts for their SQL dbs but also for us to provide a breakdown of what is blocking what.

My solution to this was to edit the datasource that contains the processblocked datapoint.  I made some good progress.  I was able to create a custom property field for the instances that I’m monitoring.  I then edited the script for the collector attribute so that in theory that if particular datapoint (processessblocked) shows a value of greater than 0, it uses hostSets() to change that custom property value to the blocker session ids using some custom sql script within. 

When I test it using the test button pointed at my sql server, it works exactly as i expected.  It shows the default value of the property unless it hits that block process value of 1 or more.  then the property reads with the needed info.  

However after I save the datasource.  And put this in to practice, no matter what the custom property on those instances never update or change.  
I also tried applying similar logic to the active discovery script with the same results.  Seems to work fine through testing but then nothing in the actual instance changes.  

I can post any or all of the groovy script if need be.  Is it something simple like some sort of syntax I’m not getting or am I not even in the right area to do what I’m attempting to do?

Any help would be appreciated and like I said, I’ll gladly provide any more info or code if need be.  

Thanks

  • Anonymous's avatar
    Anonymous
    2 years ago

    Ok, well, it can’t be through any of the built in methods since they will only allow you to output numerical data. So, unless you can somehow map the value of “blocker” to a number, your best bet will be to write a block of code that takes the value of “blocker” and either writes it as a property or the instance description or something. We’re back to this. You’ll need to write a block of code to go in the collection script to PATCH each instance with the property/value of “blocker”. 

    LM has pretty good Python and Go SDKs, but they don’t have an SDK for their native Groovy language, so you’ll have to write this all out on your own. Even the examples on their website mostly use Python instead of Groovy to access their own API. (Ug, why isn’t python natively supported in LM yet?) You might put in a ticket to support to see if they have examples in Groovy that they can point you to. I gave up trying to get Groovy to work and most of my custom datasources are written in python.

    I realize this doesn’t help that much, but it’s the direction you’ll probably have to go.

  • Testing out the final script now.  Hopefully I can post the finished code after a rousing success.

  • So the script works great for my main test device.  However the collector does keep throwing that error

    “java.net.UnknownHostException: micoresolutions.logicmonitor.com”

    When I run it on another device.  It is still the same account info so nothing should be different about it.

    As a follow up, this specifically seems to only be affecting our other two virtual machines in our training environment here at the office

  • Anonymous's avatar
    Anonymous

    As a follow up, this specifically seems to only be affecting our other two virtual machines in our training environment here at the office

    Are those devices monitored by a different collector that might have different DNS/internet path?

  • Anonymous's avatar
    Anonymous

    Is it micoresolutions or microresolutions?

  • As a follow up, this specifically seems to only be affecting our other two virtual machines in our training environment here at the office

    Are those devices monitored by a different collector that might have different DNS/internet path?

    hmmm they are monitored by their own collectors.  I am gonna go through them both to see if anything is different or maybe I messed something up in all my previous tests and failed scriptings.  

  • Anonymous's avatar
    Anonymous

    It’s as if the other collector can’t resolve the DNS name. 

  • Well, I still haven’t been able to resolve this issue with a few of our training devices.  That aside, the modified script works as intended and I finally have a solution to my original problem posed in first message of this post.  

    Huge thanks to Stuart for letting me bug him every step of the way.

    So the solution was to use API calls within int he collector attribute script inside the mysql global performance datasource.  

    I won’t share the whole script but I will post parts that seem relevant to my issue and solution.

    A lot of extra imports were needed to get this working and before the base script starts i establish to GET requests to obtain info needed to get to my final GET and PUT request 

    import groovy.sql.Sql

    import com.santaba.agent.groovyapi.http.*;

    import groovy.json.JsonSlurper;

    import org.apache.http.HttpEntity

    import org.apache.http.client.methods.CloseableHttpResponse

    import org.apache.http.client.methods.HttpGet

    import org.apache.http.client.methods.HttpPut

    import org.apache.http.impl.client.CloseableHttpClient

    import org.apache.http.impl.client.HttpClients

    import org.apache.http.util.EntityUtils

    import javax.crypto.Mac;

    import javax.crypto.spec.SecretKeySpec;

    import org.apache.commons.codec.binary.Hex;

    import groovy.json.*;

    import org.apache.http.entity.ContentType;

    import org.apache.http.entity.StringEntity;

    import org.apache.http.client.utils.URIBuilder










    // ***  Establish API Creds  ***

    def accessId = hostProps.get("lmaccess.id");

    def accessKey = hostProps.get("lmaccess.key");

    def account = 'micoresolutions';




    // *** Get Datasource ID for device as string. ***

    def resourcePath = '/device/devices/'+hostProps.get("system.deviceID")+'/devicedatasources'

    def url = "https://" + account + ".logicmonitor.com" + "/santaba/rest" + resourcePath;



        //get current time

    epoch = System.currentTimeMillis();



        //calculate signature

    requestVars = "GET" + epoch + resourcePath;



    hmac = Mac.getInstance("HmacSHA256");

    secret = new SecretKeySpec(accessKey.getBytes(), "HmacSHA256");

    hmac.init(secret);

    hmac_signed = Hex.encodeHexString(hmac.doFinal(requestVars.getBytes()));

    signature = hmac_signed.bytes.encodeBase64();



        // HTTP GET

    CloseableHttpClient httpdsid = HttpClients.createDefault();

    httpGet = new HttpGet(url);

        // Add custom filter values to GET Request

    URI uri = new URIBuilder(httpGet.getURI())

          .addParameter("fields", "id")

          .addParameter("filter", "dataSourceName:Microsoft_SQLServer_GlobalPerformance")

          .build();

          httpGet.setURI(uri);



    httpGet.addHeader("Authorization" , "LMv1 " + accessId + ":" + signature + ":" + epoch);

    response = httpdsid.execute(httpGet);

    responseBody = EntityUtils.toString(response.getEntity());

        // Turn data into id string that can be inserted into next Request

    datapull = new JsonSlurper().parseText(responseBody.toString());

    datapull =   datapull.data.items.id.value.toString()

    datapull = datapull.replace("[","")

    datasouceid = datapull.replace("]","")



    httpdsid.close()



    //  *** get instance ID for each instance on device ***



     resourcePath = "/device/devices/"+hostProps.get("system.deviceID")+"/devicedatasources/${datasouceid}/instances"

     url = "https://" + account + ".logicmonitor.com" + "/santaba/rest" + resourcePath;



        //get current time

    epoch = System.currentTimeMillis();



        //calculate signature

    requestVars = "GET" + epoch + resourcePath;



    hmac = Mac.getInstance("HmacSHA256");

    secret = new SecretKeySpec(accessKey.getBytes(), "HmacSHA256");

    hmac.init(secret);

    hmac_signed = Hex.encodeHexString(hmac.doFinal(requestVars.getBytes()));

    signature = hmac_signed.bytes.encodeBase64();



        // HTTP GET

    CloseableHttpClient httpinstanceid = HttpClients.createDefault();

    httpGet = new HttpGet(url);

    httpGet.addHeader("Authorization" , "LMv1 " + accessId + ":" + signature + ":" + epoch);

    response = httpinstanceid.execute(httpGet);

    responseBody = EntityUtils.toString(response.getEntity());

        // Male array of instance ids for final get and put request

    datapull = new JsonSlurper().parseText(responseBody.toString());

    instanceids =   datapull.data.items.id



    httpinstanceid.close()


    My final resource path --- "/device/devices/"+hostProps.get("system.deviceID")+"/devicedatasources/${datasouceid}/instances/"+instanceids

    requires a deviceID, a datasourceID and an instanceID.  So the first get pulls the correct datasource ID and the second get uses that datasourceID to grab the instance IDs.

    Then Once the collector begins its SQL query loop to find the values for all those datapoints, I insert my final GET and PUT, as well as using a custom SQL query script to find the blocker info I need.  

     if ("${counter_name}"=="Processesblocked")

                            {

    // ***Follow up if statement that decides between which Put request to use based on whether there is in fact a block or not. ***

                                if ("${cntr_value}".toInteger() > 0)

                                {

    // ***For loop to cycle through each instance on the device pulled from instance array***

                                    for(i=0; i < instanceids.size; i++)

                                    {



                                        resourcePath = "/device/devices/"+hostProps.get("system.deviceID")+"/devicedatasources/${datasouceid}/instances/"+instanceids

                                        url = "https://" + account + ".logicmonitor.com" + "/santaba/rest" + resourcePath;



                                        //get current time

                                        epoch = System.currentTimeMillis();



                                        //calculate signature

                                        requestVars = "GET" + epoch + resourcePath;



                                        hmac = Mac.getInstance("HmacSHA256");

                                        secret = new SecretKeySpec(accessKey.getBytes(), "HmacSHA256");

                                        hmac.init(secret);

                                        hmac_signed = Hex.encodeHexString(hmac.doFinal(requestVars.getBytes()));

                                        signature = hmac_signed.bytes.encodeBase64();



                                        // HTTP GET

                                        CloseableHttpClient httpinprop = HttpClients.createDefault();

                                        httpGet = new HttpGet(url);

                                        httpGet.addHeader("Authorization" , "LMv1 " + accessId + ":" + signature + ":" + epoch);

                                        response = httpinprop.execute(httpGet);

                                        responseBody = EntityUtils.toString(response.getEntity());

                                        code = response.getStatusLine().getStatusCode();

                                        datapull = new JsonSlurper().parseText(responseBody.toString());

                                        instanceprops =   datapull.data

    // ***if statement to ensure that the put request only applies to current SQL instance iteration by matching the instance displayname property with the SQL instancename ***

                                        if ("${instanceprops.displayName}"=="${instanceName}")

                                        {

                                           // ***SQL query to find blocker data***

                                            outputBlocks = runQuery("""SELECT SPID, BLOCKED, REPLACE (REPLACE (T.TEXT, CHAR(10), ' '), CHAR (13), ' ' ) AS BATCH INTO #T FROM sys.sysprocesses R CROSS APPLY sys.dm_exec_sql_text(R.SQL_HANDLE) T; WITH BLOCKERS (SPID, BLOCKED, LEVEL, BATCH) AS (SELECT SPID, BLOCKED, CAST (REPLICATE ('0', 4-LEN (CAST (SPID AS VARCHAR))) + CAST (SPID AS VARCHAR) AS VARCHAR (1000)) AS LEVEL, BATCH FROM #T R WHERE (BLOCKED = 0 OR BLOCKED = SPID) AND EXISTS (SELECT * FROM #T R2 WHERE R2.BLOCKED = R.SPID AND R2.BLOCKED <> R2.SPID) UNION ALL SELECT R.SPID, R.BLOCKED, CAST (BLOCKERS.LEVEL + RIGHT (CAST ((1000 + R.SPID) AS VARCHAR (100)), 4) AS VARCHAR (1000)) AS LEVEL, R.BATCH FROM #T AS R INNER JOIN BLOCKERS ON R.BLOCKED = BLOCKERS.SPID WHERE R.BLOCKED > 0 AND R.BLOCKED <> R.SPID ) SELECT N'    ' + REPLICATE (N'|         ', LEN (LEVEL)/4 - 1) + CASE WHEN (LEN(LEVEL)/4 - 1) = 0 THEN 'HEAD -  ' ELSE '|------  ' END + CAST (SPID AS NVARCHAR (10)) + N' ' + BATCH AS BLOCKING_TREE FROM BLOCKERS ORDER BY LEVEL ASC DROP TABLE #T;""", conn.connection, debug)



                                            blockerid = "${outputBlocks.data.blocking_tree}"



                                            instanceprops.customProperties = [[name:"blockerdetails", value:"${blockerid}"]]

                                            instanceprops = JsonOutput.toJson(instanceprops)

                                            StringEntity params = new StringEntity(instanceprops,ContentType.APPLICATION_JSON)



                                            epoch = System.currentTimeMillis(); //get current time



                                            requestVars = "PUT" + epoch + instanceprops + resourcePath;



                                            hmac = Mac.getInstance("HmacSHA256");

                                            secret = new SecretKeySpec(accessKey.getBytes(), "HmacSHA256");

                                            hmac.init(secret);

                                            hmac_signed = Hex.encodeHexString(hmac.doFinal(requestVars.getBytes()));

                                            signature = hmac_signed.bytes.encodeBase64();



                                            httpPut = new HttpPut(url);

                                            httpPut.addHeader("Authorization" , "LMv1 " + accessId + ":" + signature + ":" + epoch);

                                            httpPut.setHeader("Accept", "application/json");

                                            httpPut.setHeader("Content-type", "application/json");

                                            httpPut.setEntity(params);

                                            responsePut = httpinprop.execute(httpPut);

                                            responseBodyPut = EntityUtils.toString(responsePut.getEntity());

                                            codePut = responsePut.getStatusLine().getStatusCode();



                                        }

                                           



                                        httpinprop.close()

                                    }



                                }

                                else

                                {

                                    for(i=0; i < instanceids.size; i++)

                                    {

                                       

                                        resourcePath = "/device/devices/"+hostProps.get("system.deviceID")+"/devicedatasources/${datasouceid}/instances/"+instanceids

                                        url = "https://" + account + ".logicmonitor.com" + "/santaba/rest" + resourcePath;



                                        //get current time

                                        epoch = System.currentTimeMillis();



                                        //calculate signature

                                        requestVars = "GET" + epoch + resourcePath;



                                        hmac = Mac.getInstance("HmacSHA256");

                                        secret = new SecretKeySpec(accessKey.getBytes(), "HmacSHA256");

                                        hmac.init(secret);

                                        hmac_signed = Hex.encodeHexString(hmac.doFinal(requestVars.getBytes()));

                                        signature = hmac_signed.bytes.encodeBase64();



                                        // HTTP GET

                                        CloseableHttpClient httpinprop = HttpClients.createDefault();

                                        httpGet = new HttpGet(url);

                                        httpGet.addHeader("Authorization" , "LMv1 " + accessId + ":" + signature + ":" + epoch);

                                        response = httpinprop.execute(httpGet);

                                        responseBody = EntityUtils.toString(response.getEntity());

                                        code = response.getStatusLine().getStatusCode();

                                        datapull = new JsonSlurper().parseText(responseBody.toString());

                                        instanceprops =   datapull.data

    // ***if statement to ensure that the put request only applies to current SQL instance iteration by matching the instance displayname property with the SQL instancename ***

                                        if ("${instanceprops.displayName}"=="${instanceName}")

                                        {

                                            blockerid = "No Blocks"



                                            instanceprops.customProperties = [[name:"blockerdetails", value:"${blockerid}"]]

                                            instanceprops = JsonOutput.toJson(instanceprops)

                                            StringEntity params = new StringEntity(instanceprops,ContentType.APPLICATION_JSON)



                                            epoch = System.currentTimeMillis(); //get current time



                                            requestVars = "PUT" + epoch + instanceprops + resourcePath;



                                            hmac = Mac.getInstance("HmacSHA256");

                                            secret = new SecretKeySpec(accessKey.getBytes(), "HmacSHA256");

                                            hmac.init(secret);

                                            hmac_signed = Hex.encodeHexString(hmac.doFinal(requestVars.getBytes()));

                                            signature = hmac_signed.bytes.encodeBase64();



                                            httpPut = new HttpPut(url);

                                            httpPut.addHeader("Authorization" , "LMv1 " + accessId + ":" + signature + ":" + epoch);

                                            httpPut.setHeader("Accept", "application/json");

                                            httpPut.setHeader("Content-type", "application/json");

                                            httpPut.setEntity(params);

                                            responsePut = httpinprop.execute(httpPut);

                                            responseBodyPut = EntityUtils.toString(responsePut.getEntity());

                                            codePut = responsePut.getStatusLine().getStatusCode();

                                        }

                                        httpinprop.close()

                                    }

    Now when the collector does it’s thing it will run that SQL query and per instance, update my custom property with either the blocker details or “no blocks”.

  • Anonymous's avatar
    Anonymous

    It seems the issue has something to do with API calls in general, and if I can get anything out of support, I’ll post an update.  But for the purpose of the original post, I’d say mission accomplished.  

    I’d guess there’s a problem with those collectors accessing the internet. In postman, there’s an option on the right that looks like “</>”. If you pop that open, it’ll auto generate simple code to make your API call. If you select cURL, you should be able to paste that into a bash shell on the collector (presuming the token in that code is recent). If that fails with a similar error, that would mean a problem on the host, not related to the collector software.

  • It seems the issue has something to do with API calls in general, and if I can get anything out of support, I’ll post an update.  But for the purpose of the original post, I’d say mission accomplished.  

    I’d guess there’s a problem with those collectors accessing the internet. In postman, there’s an option on the right that looks like “</>”. If you pop that open, it’ll auto generate simple code to make your API call. If you select cURL, you should be able to paste that into a bash shell on the collector (presuming the token in that code is recent). If that fails with a similar error, that would mean a problem on the host, not related to the collector software.

    Good idea, and after testing it, it works fine through powershell and bash.  So looks like it is a collector issue.