Veeam B&R integration with New Relic (Complete technical guide)

Veeam B&R integration with New Relic (Complete technical guide)

Imagine that: I have NOC team, working with many tools to identify the problems if any.

They already have a fantastic tool which I really like, “Veeam One” and they can monitor the backup process along with many many other metrics.

So, why am I doing this?

Modern observability solutions give me a comprehensive approach. And with AI revolution, I can correlate multiple sources of incidents and accurately discover where exactly my problem is.

But of course, there are other reasons:

Silent Failures: Backup jobs can fail without triggering immediate alerts to your operations team. By the time someone checks the Veeam console, you might have missed several backup windows.

Capacity Planning Blind Spots: Storage growth trends and backup performance degradation often go unnoticed until they become critical issues. Without proper monitoring, you’re essentially flying blind on resource utilization.

Compliance Reporting Challenges: Many organizations need to demonstrate backup success rates and recovery capabilities for compliance purposes. Manual reporting from Veeam is time-consuming and error-prone.

Operational Inefficiency: Your operations team probably lives in New Relic for infrastructure monitoring. Having to context-switch to Veeam’s interface for backup status creates workflow friction and increases the likelihood of missed issues.

Although New Relic does not have an official Veeam integration. If, however, Veeam provides an API or command-line interface to get the data you want, you may use New Relic’s APIs or Flex integration to send the data to New Relic.

Understanding the Players

Veeam Backup & Replication: Your Data Protection Powerhouse

Veeam has established itself as the industry standard for backup and disaster recovery solutions, particularly in virtualized environments. At its core, Veeam provides comprehensive data protection for virtual, physical, and cloud workloads. The platform excels at creating efficient, reliable backups while offering granular recovery options that can get your business back online quickly.

What makes Veeam particularly powerful is its approach to backup verification. Every backup is automatically tested to ensure recoverability, and the platform provides detailed reporting on backup success rates, storage utilization, and recovery time objectives (RTOs). However, like many enterprise tools, Veeam’s native monitoring capabilities are primarily focused on the backup operations themselves rather than providing holistic infrastructure visibility.

New Relic: Your Observability Command Center

New Relic has evolved from a simple application performance monitoring tool into a comprehensive observability platform. It ingests telemetry data from across your entire technology stack, providing real-time insights into application performance, infrastructure health, and user experience metrics. The platform’s strength lies in its ability to correlate data from disparate sources, creating a unified view of your system’s behavior.

For infrastructure teams, New Relic offers powerful alerting capabilities, customizable dashboards, and advanced analytics that can help identify trends and potential issues before they impact business operations. The platform’s API-first approach makes it highly extensible, allowing integration with virtually any system that can generate metrics or logs.

Business Value Proposition

The integration delivers tangible business benefits that extend far beyond technical convenience. First, you significantly reduce your Mean Time to Detection (MTTD) for backup-related issues. Instead of discovering failed backups during routine checks or, worse, during a recovery scenario, your team receives immediate alerts through New Relic’s notification channels.

Second, the integration enables predictive maintenance. By analyzing backup performance trends, storage utilization patterns, and job duration metrics over time, you can identify potential issues before they impact your backup windows. This proactive approach is far more cost-effective than reactive troubleshooting.

Technical Integration Architecture

There are several approaches to integrating Veeam with New Relic, each with distinct advantages:

PowerShell-Based Integration: This method leverages Veeam’s extensive PowerShell cmdlets to extract metrics and send them to New Relic. It’s highly customizable and doesn’t require additional software components.

REST API Integration: Veeam’s Enterprise Manager provides REST APIs that can be consumed by custom applications or scripts. This approach is ideal for real-time monitoring scenarios.

Database Direct Integration: For organizations with specific requirements, directly querying Veeam’s configuration database can provide the most comprehensive data access.

Log-Based Integration: Parsing Veeam’s log files and forwarding structured data to New Relic provides detailed operational insights.

For this guide, we’ll focus on the PowerShell-based approach as it offers the best balance of functionality, maintainability, and ease of implementation.

Prerequisites and Environment Setup

Infrastructure Requirements

Before diving into the integration, ensure your environment meets these requirements:

Veeam Infrastructure: You’ll need Veeam Backup & Replication version 11 or later with PowerShell module access. The integration server should have network connectivity to your Veeam backup server and sufficient permissions to execute Veeam cmdlets.

New Relic Account: An active New Relic account with API access. You’ll need to generate an Insert API key for sending custom metrics.

Integration Server: A Windows server with PowerShell 5.1 or later, preferably with Task Scheduler access for automation. This server should have the Veeam PowerShell module installed.

Security Considerations

Security should be a primary concern when implementing this integration. Create a dedicated service account with minimal necessary permissions for running Veeam cmdlets. This account should be able to read backup job configurations and status, but shouldn’t have administrative rights to modify backup policies.

For New Relic API access, treat your Insert API key as a sensitive credential. Store it using Windows Credential Manager or a proper secrets management solution rather than hardcoding it in scripts.

Step-by-Step Implementation Guide

Step 1: Environment Preparation

Start by preparing your integration environment. On your chosen integration server, install the Veeam PowerShell module:

powershell

# Import the Veeam PowerShell module
Import-Module "C:\Program Files\Veeam\Backup and Replication\Console\Veeam.Backup.PowerShell\Veeam.Backup.PowerShell.psd1"

# Connect to your Veeam Backup & Replication server
Connect-VBRServer -Server "your-veeam-server.domain.com"

Next, verify your New Relic connectivity by testing the Events API:

powershell

$apiKey = "your-insert-api-key"
$apiUrl = "https://insights-collector.newrelic.com/v1/accounts/your-account-id/events"

$testEvent = @{
    eventType = "VeeamIntegrationTest"
    message = "Integration test successful"
    timestamp = [DateTimeOffset]::UtcNow.ToUnixTimeSeconds()
} | ConvertTo-Json

$headers = @{
    "Content-Type" = "application/json"
    "X-Insert-Key" = $apiKey
}

try {
    $response = Invoke-RestMethod -Uri $apiUrl -Method Post -Headers $headers -Body "[$testEvent]"
    Write-Host "Test event sent successfully"
} catch {
    Write-Error "Failed to send test event: $($_.Exception.Message)"
}

Step 2: Core Integration Script Development

Create the main integration script that will collect Veeam metrics and send them to New Relic:

powershell

# VeeamNewRelicIntegration.ps1

param(
    [string]$VeeamServer = "your-veeam-server.domain.com",
    [string]$NewRelicAccountId = "your-account-id",
    [string]$NewRelicApiKey = "your-insert-api-key"
)

# Import required modules
Import-Module "C:\Program Files\Veeam\Backup and Replication\Console\Veeam.Backup.PowerShell\Veeam.Backup.PowerShell.psd1"

# Configuration
$apiUrl = "https://insights-collector.newrelic.com/v1/accounts/$NewRelicAccountId/events"
$headers = @{
    "Content-Type" = "application/json"
    "X-Insert-Key" = $NewRelicApiKey
}

function Send-NewRelicEvent {
    param(
        [hashtable]$Event
    )
    
    $eventJson = $Event | ConvertTo-Json -Compress
    $body = "[$eventJson]"
    
    try {
        $response = Invoke-RestMethod -Uri $apiUrl -Method Post -Headers $headers -Body $body
        Write-Host "Event sent successfully: $($Event.eventType)"
    } catch {
        Write-Error "Failed to send event: $($_.Exception.Message)"
    }
}

function Get-VeeamJobMetrics {
    # Connect to Veeam server
    Connect-VBRServer -Server $VeeamServer
    
    # Get all backup jobs
    $jobs = Get-VBRJob
    
    foreach ($job in $jobs) {
        $lastSession = Get-VBRBackupSession -Job $job | Sort-Object EndTime -Descending | Select-Object -First 1
        
        if ($lastSession) {
            $event = @{
                eventType = "VeeamBackupJob"
                jobName = $job.Name
                jobType = $job.JobType.ToString()
                result = $lastSession.Result.ToString()
                startTime = $lastSession.CreationTime.ToString("yyyy-MM-ddTHH:mm:ssZ")
                endTime = $lastSession.EndTime.ToString("yyyy-MM-ddTHH:mm:ssZ")
                duration = ($lastSession.EndTime - $lastSession.CreationTime).TotalMinutes
                processedObjects = $lastSession.SessionInfo.Progress.ProcessedObjects
                totalObjects = $lastSession.SessionInfo.Progress.TotalObjects
                transferredSizeGB = [math]::Round(($lastSession.SessionInfo.Progress.TransferedSize / 1GB), 2)
                compressionRatio = $lastSession.SessionInfo.Progress.CompressRatio
                deduplicationRatio = $lastSession.SessionInfo.Progress.DedupRatio
                timestamp = [DateTimeOffset]::UtcNow.ToUnixTimeSeconds()
            }
            
            Send-NewRelicEvent -Event $event
        }
    }
    
    # Disconnect from Veeam server
    Disconnect-VBRServer
}

function Get-VeeamRepositoryMetrics {
    Connect-VBRServer -Server $VeeamServer
    
    $repositories = Get-VBRBackupRepository
    
    foreach ($repo in $repositories) {
        $event = @{
            eventType = "VeeamRepository"
            repositoryName = $repo.Name
            repositoryType = $repo.Type.ToString()
            totalSpaceGB = [math]::Round(($repo.GetContainer().CachedTotalSpace / 1GB), 2)
            freeSpaceGB = [math]::Round(($repo.GetContainer().CachedFreeSpace / 1GB), 2)
            usedSpaceGB = [math]::Round((($repo.GetContainer().CachedTotalSpace - $repo.GetContainer().CachedFreeSpace) / 1GB), 2)
            utilizationPercent = [math]::Round(((($repo.GetContainer().CachedTotalSpace - $repo.GetContainer().CachedFreeSpace) / $repo.GetContainer().CachedTotalSpace) * 100), 2)
            timestamp = [DateTimeOffset]::UtcNow.ToUnixTimeSeconds()
        }
        
        Send-NewRelicEvent -Event $event
    }
    
    Disconnect-VBRServer
}

# Main execution
try {
    Write-Host "Starting Veeam to New Relic integration..."
    Get-VeeamJobMetrics
    Get-VeeamRepositoryMetrics
    Write-Host "Integration completed successfully"
} catch {
    Write-Error "Integration failed: $($_.Exception.Message)"
    exit 1
}

Step 3: Advanced Metrics Collection

Expand the integration to include more detailed metrics:

powershell

function Get-VeeamPerformanceMetrics {
    Connect-VBRServer -Server $VeeamServer
    
    # Get recent backup sessions with detailed performance data
    $sessions = Get-VBRBackupSession | Where-Object { $_.CreationTime -gt (Get-Date).AddHours(-24) }
    
    foreach ($session in $sessions) {
        $taskSessions = Get-VBRTaskSession -Session $session
        
        foreach ($task in $taskSessions) {
            $event = @{
                eventType = "VeeamTaskPerformance"
                jobName = $session.JobName
                taskName = $task.Name
                vmName = $task.Name
                status = $task.Status.ToString()
                startTime = $task.Info.CreationTime.ToString("yyyy-MM-ddTHH:mm:ssZ")
                endTime = $task.Info.EndTime.ToString("yyyy-MM-ddTHH:mm:ssZ")
                duration = ($task.Info.EndTime - $task.Info.CreationTime).TotalMinutes
                avgSpeedMBps = [math]::Round(($task.Info.Progress.AvgSpeed / 1MB), 2)
                totalSizeGB = [math]::Round(($task.Info.Progress.TotalSize / 1GB), 2)
                processedSizeGB = [math]::Round(($task.Info.Progress.ProcessedSize / 1GB), 2)
                transferredSizeGB = [math]::Round(($task.Info.Progress.TransferedSize / 1GB), 2)
                compressionRatio = $task.Info.Progress.CompressRatio
                deduplicationRatio = $task.Info.Progress.DedupRatio
                timestamp = [DateTimeOffset]::UtcNow.ToUnixTimeSeconds()
            }
            
            Send-NewRelicEvent -Event $event
        }
    }
    
    Disconnect-VBRServer
}

function Get-VeeamLicenseMetrics {
    Connect-VBRServer -Server $VeeamServer
    
    $license = Get-VBRInstalledLicense
    
    $event = @{
        eventType = "VeeamLicense"
        licenseType = $license.Type.ToString()
        edition = $license.Edition.ToString()
        status = $license.Status.ToString()
        expirationDate = $license.ExpirationDate.ToString("yyyy-MM-ddTHH:mm:ssZ")
        licensedInstances = $license.LicensedInstances
        usedInstances = $license.UsedInstances
        availableInstances = ($license.LicensedInstances - $license.UsedInstances)
        utilizationPercent = [math]::Round((($license.UsedInstances / $license.LicensedInstances) * 100), 2)
        daysUntilExpiration = ($license.ExpirationDate - (Get-Date)).Days
        timestamp = [DateTimeOffset]::UtcNow.ToUnixTimeSeconds()
    }
    
    Send-NewRelicEvent -Event $event
    
    Disconnect-VBRServer
}

Step 4: Automation and Scheduling

Create a scheduled task to run the integration automatically:

powershell

# CreateScheduledTask.ps1

$taskName = "VeeamNewRelicIntegration"
$scriptPath = "C:\Scripts\VeeamNewRelicIntegration.ps1"
$taskDescription = "Automated Veeam to New Relic integration"

# Create the scheduled task action
$action = New-ScheduledTaskAction -Execute "PowerShell.exe" -Argument "-File `"$scriptPath`" -ExecutionPolicy Bypass"

# Create the scheduled task trigger (every 15 minutes)
$trigger = New-ScheduledTaskTrigger -RepetitionInterval (New-TimeSpan -Minutes 15) -RepetitionDuration (New-TimeSpan -Days 365) -At (Get-Date) -Once

# Create the scheduled task settings
$settings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries -StartWhenAvailable

# Create the scheduled task principal (run as service account)
$principal = New-ScheduledTaskPrincipal -UserId "DOMAIN\VeeamIntegrationService" -LogonType ServiceAccount

# Register the scheduled task
Register-ScheduledTask -TaskName $taskName -Action $action -Trigger $trigger -Settings $settings -Principal $principal -Description $taskDescription

Write-Host "Scheduled task '$taskName' created successfully"

Step 5: New Relic Dashboard Configuration

Create custom dashboards in New Relic to visualize your Veeam metrics:

sql

-- Backup Job Success Rate
SELECT percentage(count(*), WHERE result = 'Success') as 'Success Rate' 
FROM VeeamBackupJob 
WHERE timestamp > (unixTimestamp() - 86400)
FACET jobName

-- Storage Utilization Trend
SELECT average(utilizationPercent) as 'Utilization %' 
FROM VeeamRepository 
WHERE timestamp > (unixTimestamp() - 86400)
TIMESERIES AUTO
FACET repositoryName

-- Backup Performance Metrics
SELECT average(duration) as 'Avg Duration (minutes)', 
       average(avgSpeedMBps) as 'Avg Speed (MB/s)',
       average(compressionRatio) as 'Avg Compression Ratio'
FROM VeeamTaskPerformance 
WHERE timestamp > (unixTimestamp() - 86400)
FACET jobName

-- License Utilization Alert
SELECT latest(utilizationPercent) as 'License Utilization %',
       latest(daysUntilExpiration) as 'Days Until Expiration'
FROM VeeamLicense

Step 6: Alerting Configuration

Set up proactive alerts in New Relic:

sql

-- Failed Backup Job Alert
SELECT count(*) 
FROM VeeamBackupJob 
WHERE result != 'Success' 
AND timestamp > (unixTimestamp() - 900)

-- High Repository Utilization Alert
SELECT latest(utilizationPercent) 
FROM VeeamRepository 
WHERE utilizationPercent > 80

-- License Expiration Warning
SELECT latest(daysUntilExpiration) 
FROM VeeamLicense 
WHERE daysUntilExpiration < 30

Troubleshooting and Best Practices

Common Issues and Solutions

PowerShell Module Loading Issues: Ensure the Veeam PowerShell module is properly installed and the integration server has the necessary permissions. Sometimes, running the script as an administrator initially can resolve module loading problems.

API Rate Limiting: New Relic has rate limits on its APIs. If you’re sending high volumes of events, implement batching and retry logic with exponential backoff.

Authentication Problems: Verify that your service account has the necessary permissions to connect to Veeam and execute cmdlets. Use Test-VBRConnection to validate connectivity.

Time Zone Considerations: Ensure consistent time zone handling between Veeam and New Relic. Use UTC timestamps to avoid confusion.

Performance Optimization

Optimize your integration for better performance by implementing connection pooling, reducing API calls through batching, and caching frequently accessed data. Consider running different metric collection functions on different schedules based on how frequently the data changes.

Security Best Practices

Implement proper credential management using Windows Credential Manager or Azure Key Vault. Regularly rotate API keys and service account passwords. Monitor the integration logs for any suspicious activity or authentication failures.

Monitoring the Integration

Don’t forget to monitor the integration itself. Create alerts for integration failures, track the volume of metrics being sent to New Relic, and set up health checks to ensure the integration is running as expected.

Conclusion

Integrating Veeam with New Relic transforms your backup monitoring from a reactive to a proactive approach. You gain comprehensive visibility into your backup infrastructure, enabling better decision-making and faster incident response. The investment in setting up this integration pays dividends through improved operational efficiency and reduced risk of data loss.

Remember that this integration is not a one-time setup but an evolving component of your infrastructure that should be regularly reviewed and updated as your environment grows and changes. Start with the basic metrics, prove the value, and then gradually expand to include more sophisticated monitoring and alerting capabilities.

The combination of Veeam’s robust backup capabilities with New Relic’s powerful observability platform creates a monitoring solution that’s greater than the sum of its parts, providing the peace of mind that comes from knowing your data protection strategy is both effective and properly monitored.

Comments

No comments yet. Why don’t you start the discussion?

Leave a Reply

Your email address will not be published. Required fields are marked *