Microsoft Sentinel Logstash Output Plugin: DCR-Based Data Ingestion

Overview

Microsoft has released a public preview of a new version of the Logstash output plugin for Sentinel. made available as. This update abandons the traditional shared workspace key method and switches to the more secure Microsoft Entra ID-based application authentication. Data flow is now managed directly through Azure Monitor Data Collection Rules (DCR).

Issue: Limitations of Legacy Authentication Methods

The legacy Logstash plugin used a structure based on workspace keys. This situation created risks in terms of security policies and caused scalability problems. The new DCR-based structure closes these vulnerabilities and offers a more integrated data processing pipeline with Azure Monitor.

Solution: DCR-Based Configuration

To use the new plug-in, you must create a Microsoft Entra ID application and define the necessary permissions for it.

1. Create an application record in the Azure portal.
2. Create a 'Client Secret' for the application and note its value.
3. In your Log Analytics workspace. Create a DCR for the target table.
4. Update your Logstash configuration file (logstash.conf).

output {
  microsoft-sentinel {
    client_id => "YOUR_CLIENT_ID"
    client_secret => "YOUR_CLIENT_SECRET"
    tenant_id => "YOUR_TENANT_ID"
    dcr_immutable_id => "YOUR_DCR_ID"
    dcr_stream_name => "Custom-MyTable"
  }
}

Warning: This feature is currently in public preview. Be sure to complete your testing processes before using it in production environments.

Limitations

DCR-based ingestion method may not support all available data types. It is especially critical to define custom tables with schemas suitable for the DCR structure. Additionally, it is recommended that you monitor Logstash queues to avoid data loss when switching from the old plugin.