About the integration:

Prometheus is a powerful open-source monitoring and alerting toolkit designed for reliability and scalability. This guide walks you through the process of integrating Prometheus into your Freshservice environment


Configuration in Freshservice:


Step 1:

Head to the Admin panel, scroll to IT Operations Management, and select Monitoring Tools.



Step 2:

You are now on the Monitoring Tools list page. Select the Add monitoring tool to add a new integration.




Step 3:

You will see a list of pre-configured integrations, the gateway to custom integration using webhooks, and the option to use email as a channel for alerts. Select Prometheus.




Step 4:

A URL and auth-Key will be generated. You will require this to set up the integration in Prometheus.


Configuration in Prometheus:

  1. Ensure that you have access to the following files/applications and you have the latest version downloaded:
    • Prometheus - Download the Prometheus monitoring system version suitable for your machine
    • Alert Manager - Download the Alert Manager to configure alerting using Prometheus
    • Node Exporter - Download Node Exporter to send test alerts to Freshservice
  2. Extract the zip files and open the alert.rules.yml file in the prometheus-2.47.0.darwin-amd64 folder. Alerting rules help you define the conditions based on which the alert notifications are sent. Following is a sample alerting rule that you can add to alert.rules.yml
    groups:
- name: server_monitoring
  rules:
  - alert: ServerStatus
    expr: up == 0
    for: 2s
    labels:
      severity: '{{ if eq .Value "0" }}critical{{ else }}info{{ end }}'
    annotations:
      summary: Server status is {{ if eq .Value "0" }}down{{ else }}up{{ end }}.
      description: The server status has changed to {{ if eq .Value "0" }}down{{ else }}up{{ end }} for more than 2 seconds.
  3. Open the prometheus.yml file.This file resides the global configs that lets you specify the alerting rules that need to run. You can find more information about it here. To run the alerting rule specified in point 3, add the following code to prometheus.yml
    # Global configurationglobal:
  scrape_interval: 15s
  evaluation_interval: 15s


# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      - localhost:9093


# Load rules for evaluation
rule_files:
- 'prometheus.rules.yml'
- 'alert.rules.yml'


# Scrape configurations
scrape_configs:
- job_name: "prometheus"
  static_configs:
  - targets: ["localhost:9090"]
  - targets: ["127.0.0.1:8080"]
  4. Now, navigate to the alertmanager-0.26.0.darwin-amd64 and open the alertmanager.yml file. Here you set the end point where prometheus will send the alert. You can find more details about it here. Add the following code to alertmanager.yml 
    global:
  resolve_timeout: 5m


route:
  group_by: ['alertname', 'severity']
  group_wait: 10s
  group_interval: 1m
  repeat_interval: 3m
  receiver: 'webhook'


receivers:
- name: 'webhook'
  webhook_configs:
  - url: 'http://your-freshservice-webhook-url-here'
    send_resolved: true


templates:
- 'alertmanager/default.tmpl'


Testing the integration


1. Open the terminal on your system and navigate to the prometheus folder. Execute the following command. 

prometheus --config.file=prometheus.yml

2. Navigate to the alertmanager folder and execute the following command

./alertmanager --config.file=alertmanager.yml

3. Navigate to the Node Exporter and execute the following command

./node_exporter --web.listen-address 127.0.0.1:8080


4. Check the Prometheus and Alert Manger locally using the following links:

  • Prometheus: http://localhost:9090/alerts
  • AlertManager: http://localhost:9093/#/alerts

  • 5. To generate a critical alert, simulate a server outage by stopping the Node Exporter process in the terminal. You can then re-run the command provided in step 3 of testing the integration to bring the server back online and clear the alert