Forum Discussion

Dave_Lee's avatar
Dave_Lee
Icon for Advisor rankAdvisor
18 days ago
Solved

Extracting datapoints via API

Hi all I wonder if anyone else has found a good way to pull metric data out of LM into other systems?  I've been having a play and have something working.  My code is doing this: Query the API for...
  • Dave_Lee's avatar
    Dave_Lee
    8 days ago

    Using a report worked for me.  I create 3 different Resource Metric Trend reports for CPU, RAM, disk utilisation.  Scheduled them to run daily to generate a CSV report.  I've got about 600 servers in there at the moment, so was a few thousand rows in the Volumes report, which seems to run fine as a scheduled report and only seems to take a minute or two to run.

    I have my script call the /report/links/{reportId} endpoint, then download the most recent CSV and process that data.  This is far quicker than iterating through all the API calls needed to gather the data.

    The most granular you can go is 2 hours of data, so I'm able to get the average for each data point over the 2 hours before the report was run.

    Report frequency is 1 day at the most, but I found that I could reschedule the report from the API as well.  So, if I have the script run just after the report, it could get the average for the last 2 hours then the script can reschedule the report to run again in 2 hours.

Stuart_Weenig's avatar
Stuart_Weenig
Icon for Mastermind rankMastermind

You could cache the device/instance list and only refresh it every hour or day since it's not likely to change that much minute to minute. That would save you a little, not much, time.

You could make a dashboard widget and pull the data from there. That would give you high resolution data, based on the most recent poll by LM. /dashboard/widgets/{id}/data.

You could schedule a report then fetch the data from that report via API. That might be the fastest. You'd have to know the report ID, which wouldn't change. Then you'd have to call an undocumented API endpoint "/report/links/{reportId}" which will give you the list of links pointing to the data. Figure out which one is the most recent and pull it using the link. It wouldn't be high resolution data.

Dave_Lee's avatar
Dave_Lee
Icon for Advisor rankAdvisor
8 days ago

Using a report worked for me.  I create 3 different Resource Metric Trend reports for CPU, RAM, disk utilisation.  Scheduled them to run daily to generate a CSV report.  I've got about 600 servers in there at the moment, so was a few thousand rows in the Volumes report, which seems to run fine as a scheduled report and only seems to take a minute or two to run.

I have my script call the /report/links/{reportId} endpoint, then download the most recent CSV and process that data.  This is far quicker than iterating through all the API calls needed to gather the data.

The most granular you can go is 2 hours of data, so I'm able to get the average for each data point over the 2 hours before the report was run.

Report frequency is 1 day at the most, but I found that I could reschedule the report from the API as well.  So, if I have the script run just after the report, it could get the average for the last 2 hours then the script can reschedule the report to run again in 2 hours.