Azure - Customize Azure Log Search alerts using Automation Runbook
Introduction
As you might know the Log Search Alert – V2 alert has been released and it’s rolled out globally. The v1 alerts were quite easy understand interms of what went wrong, but the v2 alerts have more information which can be ignored for the people who's working as NOC.
By default this cannot be customized as it's not that matured enough to do that, so i have created this runbook which does the customization for v1 & v2 alerts.
Benefits of this custom alert is,
- Both V1 & V2 alert is covered as part of this custom alert. All the existing alerts can also be routed through this custom script.
- All types of alerts like CPU, Disk, Memory & Event are supported by this custom alert.
- Additionally, Start & EndTime of the event, Search query, Fired data time & etc.. parameters are enabled for this custom alert.
- Finally, the Subject, Body of the email & affected details has been changed.
- SendGrid or any email solution
- Webhook from the Automation Runbook
- Finally the alerts
Create the action groups and add the webhook which we created earlier,
Create the alert and add the action group which we created.
That's all, customization is done and now it's ready to test.
Default v2-Alert: If you see the below picture from the default alert it's incorrect, because the OMS workspace is mentioned as Affected Resource, which will make confuse for everyone.
Customized v1-Alert:
Customized v2-Alert:
Runbook Script:
param (
[Parameter(Mandatory=$false)]
[object] $WebhookData
)
$servicePrincipalConnection = Get-AutomationConnection -Name AzureRunAsConnection
$params = @{
ServicePrincipal = $true
TenantId = $servicePrincipalConnection.TenantId
ApplicationId = $servicePrincipalConnection.ApplicationId
CertificateThumbprint = $servicePrincipalConnection.CertificateThumbprint
}
Add-AzAccount @params
if ($WebhookData) {
write-output "Webhook data $WebhookData"
# Logic to allow for testing in Test Pane
if(-Not $WebhookData.RequestBody){
$WebhookData = (ConvertFrom-Json -InputObject $WebhookData)
write-output "test Webhook data $webhookData"
}
}
$jsonrequest = ConvertFrom-Json -InputObject $webhookData.RequestBody
$alertcontext = $jsonrequest.data.alertContext
$Metricvalue = $alertcontext.condition.allof.metricValue
$ComputerName = $jsonrequest.data.essentials.configurationItems | % { ($_ | Out-String).Trim() -replace '@\{|\}' }
$MonitoringType = $jsonrequest.data.essentials.monitoringService
$sev = "Warning"
if ($MonitoringType -eq "Log Alerts V2")
{
$Dimensionresults = $alertcontext.condition.allof.dimensions
$strresult = $Dimensionresults.value | Out-String
$strresult += $Metricvalue
$AffectedDetails = $strresult
$message += "<H2> Log Search Alert for $ComputerName </H2><br><br><table border=1><tr><th> Name </th><th> Description </th></tr>"
$message += "<tr><td> Alert Name </td><td>"+$jsonrequest.data.essentials.alertRule + "</td></tr>"
$message += "<tr><td> Description </td><td> "+$jsonrequest.data.essentials.description + "</td></tr>"
$message += "<tr><td> Affected Details </td><td> "+$AffectedDetails + "</td></tr>"
$message += "<tr><td> Severity </td><td>"+$sev + "</td></tr>"
$message += "<tr><td> Evaluation Start Time </td><td>"+$jsonrequest.data.alertContext.condition.windowStartTime + "</td></tr>"
$message += "<tr><td> Evaluation End Time </td><td>"+$jsonrequest.data.alertContext.condition.windowEndTime + "</td></tr>"
$message += "<tr><td> Fired DateTime </td><td> "+$jsonrequest.data.essentials.firedDateTime + "</td></tr>"
$message += "<tr><td> Monitoring Service Type </td><td>"+$jsonrequest.data.essentials.monitoringService + "</td></tr>"
$message += "<tr><td> SignalType </td><td>"+$jsonrequest.data.essentials.SignalType + "</td></tr>"
$message += "<tr><td> Search Query </td><td>"+$alertcontext.condition.allof.searchQuery + "</td></tr>"
$message += "</table>"
write-output "Message is $message"
} else {
$Searchresultscolumns = $jsonrequest.data.alertContext.SearchResults.tables.Columns
$Searchresultsrows = $jsonrequest.data.alertContext.SearchResults.tables.rows
$columns = foreach($column in $Searchresultscolumns)
{
$properties = [Ordered]@{}
for($i=0; $i -lt $Searchresultscolumns.Length; $i++)
{
$value = $Searchresultsrows[$i]
$name = $Searchresultscolumns[$i].Name
$properties[$name] = $value
}
[PSCustomObject]$properties
}
write-output "This is final result of columns"$columns[0]
$message += "<H2> Log Search Alert for $ComputerName </H2><br><br><table border=1><tr><th> Name </th><th> Description </th></tr>"
$message += "<tr><td> Alert Name </td><td>"+$jsonrequest.data.essentials.alertRule + "</td></tr>"
$message += "<tr><td> Description </td><td> "+$jsonrequest.data.essentials.description + "</td></tr>"
$message += "<tr><td> Affected Details </td><td> "+$columns[0] + "</td></tr>"
$message += "<tr><td> Severity </td><td>"+$sev + "</td></tr>"
$message += "<tr><td> Evaluation Start Time </td><td>"+$jsonrequest.data.alertContext.SearchIntervalStartTimeUtc + "</td></tr>"
$message += "<tr><td> Evaluation End Time </td><td>"+$jsonrequest.data.alertContext.SearchIntervalEndtimeUtc + "</td></tr>"
$message += "<tr><td> Fired DateTime </td><td> "+$jsonrequest.data.essentials.firedDateTime + "</td></tr>"
$message += "<tr><td> Monitoring Service Type </td><td>"+$jsonrequest.data.essentials.monitoringService + "</td></tr>"
$message += "<tr><td> SignalType </td><td>"+$jsonrequest.data.essentials.SignalType + "</td></tr>"
$message += "<tr><td> Search Query </td><td>"+$jsonrequest.data.alertContext.searchQuery + "</td></tr>"
$message += "</table>"
write-output "Message is $message"
}
$Subject = "Azure |"+$jsonrequest.data.essentials.alertRule
$EmailTo = "test@email.com"
$EmailFrom = "alerts@azure.com"
$keyVaultName = "keyvault name"
$ApiKeyName = "keyvault key name"
$MailParameters = @{
Subscription = $KVSubscription.Trim()
VaultName = $keyVaultName.Trim()
ApiKeyName = $ApiKeyName.Trim()
destEmailAddress = $EmailTo
fromEmailAddress = $EmailFrom
subject = $Subject
bodyContent = $message
}
Send-AzMail @MailParameters
#Code ends
#Send-AzMail command is customized and might not be available publically.
Comments