Microsoft Azure Event Hub

Event Hubs is a fully managed, real-time data ingestion service that is simple, trusted, and scalable. Stream millions of events per second from any source to build dynamic data pipelines and immediately respond to business challenges.

Why might you use Azure Event Hubs with FortiSIEM?

You can aggregate all your custom log sources and telemetry data to Azure Event Hubs, which FortiSIEM can then ingest and process provided there is a FortiSIEM parser created for those custom log sources.

Although FortiSIEM already ingests Azure audit logs via another integration, Azure has an option to send diagnostic Audit logs to Azure Event Hub, that FortiSIEM can then ingest.

https://docs.microsoft.com/en-us/azure/active-directory/reports-monitoring/tutorial-azure-monitor-stream-logs-to-event-hub

You can also export Azure monitoring data to an event hub: -- See section “Monitoring Data Available” for information on log types that can be exported to Azure Event Hub.

https://docs.microsoft.com/en-us/azure/azure-monitor/essentials/stream-monitoring-data-event-hubs

FortiSIEM uses the Azure Python SDK to integrate logs from the event hub to perform comprehensive security analysis. Azure Log Integration simplifies the task of integrating Azure logs with your on-premises SIEM system. The recommended method for integrating Azure logs is to stream the logs into event hubs via the Azure Monitor. FortiSIEM provides a connector to further integrate logs from the event hub into the SIEM.

Azure produces extensive logging for each Azure service. The logs represent these log types:

• Control/management logs: Provide visibility into the Azure Resource Manager CREATE, UPDATE, and DELETE operations. An Azure activity log is an example of this type of log.
• Data plane logs: Provide visibility into events that are raised when you use an Azure resource. An example of this type of log is the Windows Event Viewer's System, Security, and Application channels in a Windows virtual machine. Another example is Azure Diagnostics logging, which you configure through Azure Monitor.
• Processed events: Provide analyzed event and alert information that are processed for you. An example of this type of event is Azure Security Center alerts. Azure Security Center processes and analyzes your subscription to provide alerts that are relevant to your current security posture.

For more information on how to stream any type of log to an event hub, see:

https://docs.microsoft.com/en-us/azure/azure-monitor/platform/stream-monitoring-data-event-hubs

• What is Discovered and Monitored
• Event Types
• Reports
• Rules
• Configuration in Azure
• Configuration in FortiSIEM
• Sample Events

What is Discovered and Monitored

Protocol	Information Discovered	Information Collected	Used For
Azure Python SDK	None	Audit Logs	Security Monitoring

Event Types

No defined event types.

Reports

No defined reports.

Rules

No defined rules.

Configuration in Azure

Create an Event Hub Namespace and Event Hub

Complete these steps in the Azure Portal:

Step 1: Create a Resource Group in Azure

A resource group is a logical collection of Azure resources. All resources are deployed and managed in a resource group. To create a resource group:

1. Login to the Azure portal: https://portal.azure.com/
2. Click Resource groups in the left navigation pane.
3. Click Add.
4. For Subscription, select the name of the Azure subscription in which you want to create the resource group.
5. Enter a unique name for the resource group, The system immediately checks to see if the name is available in the currently selected Azure subscription.
6. Select a Region for the resource group.
7. Click Review + Create.
8. Click Create on the Review + Create page.

Note: In the example used in step 2, a Resource Group called fsm1 was created.

Step 2: Create an Event Hub Namespace

An Event Hub namespace provides a unique scoping container, referenced by its fully-qualified domain name, in which you create one or more event hubs. To create a namespace in your resource group using the portal, complete the following steps:

1. In the Azure portal, click Create a resource at the top left of the screen.

   

2. In the “Search the Market text box, enter Select All services in the left menu, select star (*) next to Event Hubs, and then click the Create button in the ANALYTICS category.

   

3. On the Create namespace page, complete the following steps:
   1. Enter a name for the namespace. The system immediately checks to see if the name is available.
   2. Choose the pricing tier (Basic or Standard).
   3. Select the subscription in which you want to create the namespace.
   4. Select a location for the namespace.
   5. Click Create. You may have to wait a few minutes for the system to fully provision the resources.

      

4. Refresh the Event Hubs page to see the event hub namespace. You can check the status of the event hub creation in the alerts.
5. Select the namespace. You see the home page for your Event Hubs Namespace in the portal.

Step 3: Create an Event Hub

To create an event hub within the namespace, follow these steps:

1. In the Event Hubs Namespace page, click Event Hubs in the left menu.

   

2. At the top of the window, click + Event Hub.
3. Enter a name for your event hub, then click Create.

   

4. You can check the status of the event hub creation in alerts. After the event hub is created, you see it in the list of event hubs.

Step 4: Configure an Event Hub Namespace

1. Select an event hub namespace and go to Shared access policies, and then click +Add. Enter the Policy name, check the Listen box, and then click Create.
   
   Select one of the Shared Access policies just created.
2. The Azure Python SDK needs the SAS Policy name (defined in step 4.1) and the Primary key when creating the credential in FortiSIEM. Copy the primary key and policy name to a text editor for later use.

   

   Note: When the event hub namespace is created, Azure will also create a default Shared Access Policy named RootManageSharedAcessKey.

3. Select an event hub namespace and go to Event Hubs.
4. Select an event hub and go to Consumer group. You can click +Consumer group or use default group name $default.

Note: If you have selected Basic (1 Consumer Group), then there will be no option to add a another Consumer group.

Configuration in FortiSIEM

Complete these steps in the FortiSIEM UI:

1. Go to the ADMIN > Setup > Credentials tab.
2. In Step 1: Enter Credentials:
   1. Follow the instructions in "Setting Credentials" in the User's Guide to create a new credential.
   2. Enter these settings in the Access Method Definition dialog box and click Save:

      Settings	Description
      Name	Enter a name for the credential
      Device Type	Microsoft Azure Event Hub
      Access Protocol	AZURE PYTHON SDK
      Pull Interval	The interval in which FortiSIEM will pull events from Azure Event Hub. Default is 5 minutes.
      Event Hub Namespace	The name of the Azure event hub namespace
      Event Hub Name	The name of the Azure event hub.
      SAS Policy Name	Shared Access (SAS) Policy Name
      Primary Key	The name of the primary key
      Consumer Group	The name of the consumer group
      Description	Description of the device

      Based on the example screenshots, this is the configuration in FortiSIEM:

      

3. In Step 2: Enter IP Range to Credential Associations, click New.
   1. Enter a host name, an IP, or an IP range in the IP/Host Name field. For this integration, enter "azure.com".
   2. Select the name of your Azure event hub credential from the Credentials drop-down list.
   3. Click Save.

      

4. Click the Test drop-down list and select Test Connectivity to test the connection to Azure event hub.
5. To see the jobs associated with Azure, select ADMIN > Setup > Pull Events.
6. To see the received events select ANALYTICS, then enter "Azure" in the search box.

Note: Azure services must be configured to write to the Event Hub before there are any events to be collected.

Sample Events

{"records": [{ "count": 0, "total": 0, "minimum": 0, "maximum": 0, "average": 0, "resourceId": "/SUBSCRIPTIONS/3ED4EE1C-1A83-4E02-A928-7FF5E0008E8A/RESOURCEGROUPS/USER_TEST/PROVIDERS/MICROSOFT.EVENTHUB/NAMESPACES/FORTISIEMEVENTHUB", "time": "2019-02-21T05:21:00.0000000Z", "metricName": "EHAMBS", "timeGrain": "PT1M"},{ "count": 0, "total": 0, "minimum": 0, "maximum": 0, "average": 0, "resourceId": "/SUBSCRIPTIONS/3ED4EE1C-1A83-4E02-A928-7FF5E0008E8A/RESOURCEGROUPS/USER_TEST/PROVIDERS/MICROSOFT.EVENTHUB/NAMESPACES/FORTISIEMEVENTHUB", "time": "2019-02-21T05:22:00.0000000Z", "metricName": "EHAMBS", "timeGrain": "PT1M"},{ "count": 0, "total": 0, "minimum": 0, "maximum": 0, "average": 0, "resourceId": "/SUBSCRIPTIONS/3ED4EE1C-1A83-4E02-A928-7FF5E0008E8A/RESOURCEGROUPS/USER_TEST/PROVIDERS/MICROSOFT.EVENTHUB/NAMESPACES/FORTISIEMEVENTHUB", "time": "2019-02-21T05:23:00.0000000Z", "metricName": "EHAMBS", "timeGrain": "PT1M"},{ "count": 0, "total": 0, "minimum": 0, "maximum": 0, "average": 0, "resourceId": "/SUBSCRIPTIONS/3ED4EE1C-1A83-4E02-A928-7FF5E0008E8A/RESOURCEGROUPS/USER_TEST/PROVIDERS/MICROSOFT.EVENTHUB/NAMESPACES/FORTISIEMEVENTHUB", "time": "2019-02-21T05:24:00.0000000Z", "metricName": "EHAMBS", "timeGrain": "PT1M"}]}