Microsoft Office 365 Audit

FortiSIEM Support added: 4.8.1

FortiSIEM last modification: 7.1.0

Vendor: Microsoft

Vendor version tested: Not Provided

Product Information: https://www.microsoft.com/en-us/microsoft-365/business

• What is Monitored

• Configuring Office365 Auditing

• Office365 Auditing Rules

• Office365 Auditing Reports

• Office365 Auditing Dashboard

• Sample Events for Office 365 Audit Activity

• Enabling Mailbox Auditing

What is Monitored

Office 365 Management Activity API (manage.office.com) – The Office 365 Management Activity API aggregates actions and events into tenant-specific content blobs, which are classified by the type and source of the content they contain. Currently, the following content types are supported:

• Audit.AzureActiveDirectory

• Audit.Exchange

• Audit.SharePoint

• Audit.General (includes all other workloads not included in the previous content types)

• DLP.All (DLP events only for all workloads)

For details about the events and properties associated with these content types, see Office 365 Management Activity API schema. An extensive list of Office 365 services are audited via this method, and service names can be seen by inspecting the “AuditLogRecordType” of this link: https://learn.mcrosoft.com/en-us/office/office-365-management-api/office-365-management-activity-api-schema

The following activities are monitored:

• File and folder activities

• Sharing and access request activities

• Site administration activities

• Exchange mailbox activities

• User administration activities

• Group administration activities

• Application administration activities

• Role administration activities

Microsoft Graph API (graph.microsoft.com) – This API is used to obtain Office 365 Mail Statistics. The following activity report is run: https://graph.microsoft.com/v1.0/reports/getEmailActivityCounts(period='D30')

Office 365 Reporting Web Service (reports.office365.com) – This API is used to obtain Office 365 Mail Statistics. The following reporting web service reports are run:

• https:// reports.office365.com/ecp/reportingwebservice/reporting.svc/MailDetailATP

• https:// reports.office365.com/ecp/reportingwebservice/reporting.svc/MailTrafficSummary (TopMailSender, TopMailRecipient, and TopMalwareRecipient categories)

The following information is obtained:

• Email Activity Reports

• Top Mail Senders

• Top Mail Receivers

• Top Malware Recipients

• All Mail sent/received statistics

Configuring Office365 Auditing

Office365 Auditing configuration is comprised of three main steps. Before proceeding, ensure you have a good understanding of mailbox auditing. Supplemental information is provided in Enabling Mailbox Auditing.

• Step 1: Configure Office365 Mailboxes for Auditing

• Step 2: Configure Office365 for Auditing by FortiSIEM

• Step 3: Configure FortiSIEM

Step 1: Configure Office365 Mailboxes for Auditing

To set up mailbox auditing in Office 365, take the following steps.

1. Login to Exchange Online (Note: Instructions differ slightly for GCC High and other gov cloud Organizations)

2. Powershell Command:
   

   Connect-ExchangeOnline -UserPrincipalName navin@contoso.onmicrosoft.com

3. For each mailbox you'd like to turn on auditing, run the following command:

   Set-Mailbox -Identity <MailboxIdentity> -AuditEnabled $true

   • To bulk set for every mailbox run:

     Get-Mailbox -ResultSize Unlimited -Filter{RecipientTypeDetails -eq "UserMailbox"} | Set-Mailbox -AuditEnabled $true

Important Notes

• Although mailbox audit logging is on by default for all organizations, only users with licenses that include Audit (Premium) (collectively referred to in this article as E5/A5/G5 licenses) return mailbox audit log events in audit log searches in the Microsoft Purview compliance portal or via the Office 365 Management Activity API by default.

• Mailbox Operations that require premium licenses: ( Available only for users with E5/A5/G5 licenses.)
  

  Mailbox action	Description
  SearchQueryInitiated	A person uses Outlook (Windows, Mac, iOS, Android, or Outlook on the web) or the Mail app for Windows 10 to search for items in a mailbox.
  MailItemsAccessed	Occurs when mail data is accessed by mail protocols and clients.
  MessageBind	A message was viewed in the preview pane or opened by an admin.
  ***Send	The user sends an email message, replies to an email message, or forwards an email message.

  Note: Remember, an admin with Full Access permission to a mailbox is considered a delegate.

• Mailbox Audit Operations that do not have auditing turned on by default:
  

  Mailbox action	Description
  Copy (Admin only)	A message was copied to another folder.
  FolderBind (Admin and Delegate)	A mailbox folder was accessed. This action is also logged when the admin or delegate opens the mailbox.
  MailboxLogin (Owner only)	The user signed into their mailbox.
  MessageBind (Admin only)	A message was viewed in the preview pane or opened by an admin. (not applicable to E5/A5/G5 licenses)
  Move	A message was moved to another folder.
  RecordDelete	An item that's labeled as a record was soft-deleted (moved to the Recoverable Items folder). Items labeled as records can't be permanently deleted (purged from the Recoverable Items folder).
  SearchQueryInitiated (Owner only)	A person uses Outlook (Windows, Mac, iOS, Android, or Outlook on the web) or the Mail app for Windows 10 to search for items in a mailbox.
  UpdateComplianceTag	A retention label was updated.

Step 2: Configure Office365 for Auditing by FortiSIEM

Take the following steps to configure Office365 for Auditing by FortiSIEM.

• Create Office 365 Credential

• Create New App Registration

• Generate Secret Key

• Configure API Permissions for Application Registration

• Grant Sharepoint Permissions

• Grant Graph API Permissions

• Grant Reports API Permissions

Create Office 365 Credential

1. Login to the Azure Portal.

2. Go to Microsoft Entra ID (Formerly Azure AD).

3. On the left hand toolbar, select App registrations.

Create New App Registration

1. Click New registration.

2. In the Name field, enter "FortiSIEM App".

3. From Supported Account Types, select Accounts in this organizational directory only (<domain> only – single tenant).

4. Leave the Redirect URI (optional) field blank.

5. Click Register.
   

   After clicking Register, on the redirected page showing your registration details, note the following:

   • Application (Client) ID

   • Directory (Tenant) ID

Generate Secret Key

To generate a secret key, take the following steps:

1. On the left hand toolbar, click Certificates and Secrets.

2. Click New client secret.

3. In the right side popup, enter the following information:

   • In the Description field, enter "FortiSIEM Secret Key".

   • From the Expires drop-down list, select 730 days (24 months) or your desired expiration.
     

     Note: You must update the key in FortiSIEM configuration BEFORE it expires by creating a new key before retiring the old one.

4. Click Add.
   

5. Record the new API Key secret value at the bottom of the page. You can only view this once and will need it later for configuring FortiSIEM, so store the information in a secure location for configuration later.

Configure API Permissions for Application Registration

1. In the left pane, navigate to API permissions.

2. Click Add a permission.

3. Select Office 365 Management APIs.

4. Click Application permissions and expand all.

5. Select all permissions with "Read" access. (There is no reason to write).

6. Click Add permissions.

   You will see a warning: "Permissions have changed." Users and/or admins will have to consent even if they have already done so previously.

Grant Sharepoint Permissions

For Sharepoint Permissions, take the following step:

Add permission > Sharepoint > Application Permissions > Sites.Read.All

Grant Graph API Permissions

For Graph API Permissions, take the following steps:

1. Login to Azure Portal.

2. Go to Microsoft Entra ID (Formerly Azure AD).

3. On the left hand toolbar, select App registrations.

4. Select your app from the list if it is already defined.

5. On the left hand toolbar, select API permissions.

6. Select Microsoft Graph > Application permissions.

7. Select the following permissions:

   • Reports.Read.All (Primary)

   • IdentityRiskEvent.Read.All

   • IdentityRiskyServicePrincipal.Read.All

   • IdentityRiskyUser.Read.All

   • SecurityEvents.Read.All

   • SecurityIncident.Read.All (Microsoft365 Defender APIs)

   • ThreatIndicators.Read.All

   • ThreatIntelligence.Read.All

   • User.Read.All

Grant Reports API Permissions

For Office 365 Reports API Permissions, take the following steps:

1. Login to Azure Portal.

2. Go to Microsoft Entra ID (Formerly Azure AD).

3. On the left hand toolbar, select App registrations.

4. Select your app from the list if it is already defined.

5. On the left hand toolbar, select API permissions.

6. Click Add a permission.

7. Select the middle tab at the top labeled APIs my organization uses.

8. In the search bar, enter "Office 365 Exchange Online" and select it (Note: It does not appear by default).

9. Select Application Permissions.

10. Locate the ReportingWebService drop-down list and select the permission ReportingWebService.Read.All.

11. Make sure to click the Grant admin consent for <tenant> after saving. Click grant admin consent and select Yes when you see the prompt: Do you want to grant consent for the requested permissions for all accounts in your_organization? alert. This will update any existing admin consent records this application already has to match what has been configured.

Step 3: Configure FortiSIEM

FortiSIEM configuration is comprised of two main steps.

• Define Office 365 Management Credential

• Create IP Range to Credential Association and Test Connectivity

Define Office 365 Management Credential

Log in to the FortiSIEM Supervisor node, and take the following steps.

1. Go to the ADMIN > Setup > Credentials tab.
2. In Step 1: Enter Credentials:
   1. Follow the instructions in “Setting Credentials“ in the User's Guide to create a new credential.
   2. Enter these settings in the Access Method Definition dialog box and click Save:
      

Settings	Description
Name	Enter a name for the credential
Device Type	Microsoft Office365
Access Protocol	Office 365 Mgmt Activity API
Tenant ID	Use the ID from Azure Login URL. See Step 5 in Create Office 365 API Credential.
Password config	If you select Manual, take the following steps: For Client ID, use the value obtained from the Create New App Registration step in Create Office 365 Credential. For Client Secret, use the value obtained from the Generate Secret Key step in Create Office 365 Credential. For CyberArk SDK credential method, see CyberArk SDK Password Configuration. For CyberARK REST API credential method, see CyberArk REST API Password Configuration.
Authentication Endpoint	Enter the authentication endpoint. The Authentication Endpoint depends on the type of Office 365 environment you have: Enterprise plan: login.windows.net GCC government plan: login.microsoftonline.com GCC High government plan: login.microsoftonline.us DoD government plan: login.microsoftonline.us Note: Do NOT include "https://" in the Authentication Endpoint URL field.
Authentication Protocol	Enter the token location. For example, /oauth2/token.
Organization	The organization the device belongs to.
Description	Description of the device.

Create IP Range to Credential Association and Test Connectivity

From the FortiSIEM Supervisor node, take the following steps.

1. In Step 2: Enter IP Range to Credential Associations, if the organization has more than 1 collector, select the collector from the drop-down list that will do the API polling. If the organization has 1 or no collectors, there is no drop-down and you can proceed to the next step.
2. Click New to create a new association.
   1. Select the name of the credential created in the Define Office 365 Management Credential from the Credentials drop-down list.
   2. In the IP/Host Name field, enter the API Endpoint based off your Office 365 plan type. Your options are:
      

      • Enterprise plan: manage.office.com

      • GCC government plan: manage-gcc.office.com

      • GCC High government plan: manage.office365.us

      • DoD government plan: manage.protection.apps.mil

   3. Click Save.
3. Select the entry just created and click the Test drop-down list and select Test Connectivity without Ping to start the polling. A pop up will appear and show the Test Connectivity results.
4. Go to ADMIN > Setup > Pull Events and make sure an entry is created for Office 365 Log Collection.

Office365 Auditing Rules

The following are rules all generated via datasource : Office365 Mgmt Activity API. Note that the following are examples, and non-exhaustive. Navigate to Resources >Rules and search for "Office365:" in the main Search... field to see available rules.

• Office365: Abnormal Logon Detected

• Office365: Admin or Delegated User Created Mailbox Forwarding Rule for another User

• Office365: Brute Force Login Attempts - Same Source

• Office365: Brute Force Login Attempts - Same User

• Office365: Brute Force Logon Success

• Office365: Delete Message Inbox Rule Created

• Office365: Identity Protection Detected a Risky User or SignIn Activity

• Office365: Mailbox Login from Outside My Country

• Office365: Mailbox SendAs or SendOnBehalf has occurred

• Office365: Mailbox User Created Mail Forwarding Rule

• Office365: Move To Folder Inbox Rule Created

• Office365: Set-Mailbox Forwarding Action Created

• Office365: Strong Authentication Disabled for a User

• Office365: Suspicious File Type Uploaded

Office365 Auditing Reports

The following are reports all generated via datasource : Office365 Mgmt Activity API. Note that the following are examples, and non-exhaustive. Navigate to Resources > Reports and search for "Office365:" in the main Search... field to see available reports.

• Office365: Top Entra ID Logons by Source

• Office365: Top Entra ID Failed User Logon

• Office365: Top Mailbox Logins by Source Country

• Office365: Top Mailbox Logins by Source IP

• Office365: Top Mail Senders

• Office365: Top Mail Recipients

• Office365: Top SendAs or SendOnBehalf Activity by Originating User

• Office365: Top Sharepoint Links Used

• Office365: Top Sharepoint Secure Links Used by Source

• Office365: Top Sharepoint Company Links Used by Source

• Office365: Top Sharepoint Anonymous Links Used by Source

Office365 Auditing Dashboard

The Office 365 Dashboard contains 4 tabs:

• Logon Audit

• Sharepoint Audit

• Sensitivity Label Audit

• Mail Statistics

Logon Audit

Object	Description
Data Source	Microsoft Office365 Management Activity API - Collects both Entra ID (formerly Azure AD) logon events as well as Exchange mailbox logons.
Widgets	Various mailbox and Entra ID logon success/failure activity reports.
Troubleshooting	Ensure that mailbox auditing is enabled for every mailbox in your organization.

Sharepoint Audit

Object	Description
Data Source	Microsoft Office365 Management Activity API
Widgets	These are reports based on Sharepoint shared link activity primarily, others may be added in the future. Other Sharepoint reports are available.

Sensitivity Label Audit

This is only populated if you are using Microsoft MIP for Exchange to assign sensitivity labels to emails.

Object	Description
Data Source	Microsoft Office365 Management Activity API
Reference Documents	https://support.microsoft.com/en-us/office/apply-sensitivity-labels-to-your-files-and-email-2f96e7cd-d5a4-403b-8bd7-4cc636bae0f9 https://techcommunity.microsoft.com/t5/security-compliance-and-identity/announcing-new-microsoft-information-protection-capabilities-to/ba-p/1999692

Mail Statistics

This is mail statistical data such as top senders / receivers collected from primarily the Office365 Reporting Web Service and Graph APIs.

Object	Description
Data Source	Office365 Reporting Web Service and Microsoft Graph Reporting API

Sample Events for Office 365 Audit Activity

[MS_OFFICE365_MessageTrace] = {"EndDate":"/Date(1694294532249)/","FromIP":"10.10.10.19","Index":3,"MessageId":"<11111111-855c-44ff-8edb-ffaffae245dd@example.com>","MessageTraceId":"11111111-e22c-485e-be92-08dbb082b92e","Organization":"example.onmicrosoft.com","Received":"/Date(1694187970214)/","RecipientAddress":"example@azurestorefortinet.onmicrosoft.com","SenderAddress":"example@microsoft.com","ServerHostName":"reports.office365.com","Size":193926,"StartDate":"/Date(1694121732249)/","Status":"Resolved","Subject":"We detected synchronization errors in your directory","TenantId":"11111111-1b14-42a1-8dcf-4b21dece61ba","ToIP":null,"__metadata":{"id":"https://reports.office365.com/ecp/ReportingWebService/Reporting.svc/MessageTrace(3)","type":"TenantReporting.MessageTrace","uri":"https://reports.office365.com/ecp/ReportingWebService/Reporting.svc/MessageTrace(3)"},"phCustId":1}

[MS_OFFICE365_MailTrafficSummary_TopMalwareRecipient] = {"ServerHostName":"reports.office365.com","ServerIpAddr":"10.10.10.10","TenantId":"0000-0000-00000-0000-00000","TopMalwareRecipient":"","count":"0","phCustId":1}

[MS_OFFICE365_MailTrafficSummary_Count] = {"Meeting Created":"41","Meeting Interacted":"57","Read":"12458","Report Date":"2023-08-25","Report Period":"30","ServerHostName":"reports.office365.com","ServerIpAddr":"10.10.10.10","TenantId":"0000-0000-00000-0000-00000","count":"1911","direction":"outbound","phCustId":1,"\ufeffReport Refresh Date":"2023-09-23"}

[MS_OFFICE365_MessageTrace] = {"Action":"Hosted Mailbox: Inbox","BulkComplaintLevel":null,"Date":"/Date(1695940243000)/","Direction":"Inbound","Domain":"example1.com","EndDate":"/Date(-62135596800000)/","EventType":"Message passed","FileHash":null,"FileName":null,"Index":4,"MalwareName":null,"MessageId":"<9DB1A48827764D1783C1CD22F9BC8D5B@MUMTILLPCMS>","MessageTraceId":"125227f2-a7fe-42ec-b748-08dbc0728a18","Organization":".com","RecipientAddress":"user@example.com","SenderAddress":"user1@example1.com","ServerHostName":"reports.office365.com","ServerIpAddr":"10.10.10.10","StartDate":"/Date(-62135596800000)/","Subject":"Compliance under Maharashtra State Tax On Professions, Trades, Callings And Employments Act,1975 - ","TenantId":"0000-0000-00000-0000-00000","VerdictSource":"NotSpam","__metadata":{"id":"https://reports.office365.com/ecp/ReportingWebService/Reporting.svc/MailDetailATP(4)","type":"TenantReporting.MailDetailATPReport","uri":"https://reports.office365.com/ecp/ReportingWebService/Reporting.svc/MailDetailATP(4)"},"phCustId":1}

Enabling Mailbox Auditing

Note: The following is an excerpt of the Microsoft Manage mailbox auditing article here.

Mailbox audit logging is turned on by default in all organizations. This effort started in January 2019, and means that certain actions performed by mailbox owners, delegates, and admins are automatically logged.

Here are some benefits of mailbox auditing on by default:

• Auditing is automatically turned on when you create a new mailbox. You don't need to manually turn on mailbox auditing for new users.

• You don't need to manage the mailbox actions that are audited. A predefined set of mailbox actions are audited by default for each sign-in type (Admin, Delegate, and Owner).

• When Microsoft releases a new mailbox action, the action might be added automatically to the list of mailbox actions that are audited by default (subject to the user having the appropriate license). This result means you don't need to add new actions on mailboxes as they're released.

• You have a consistent mailbox auditing policy across your organization (because you're auditing the same actions for all mailboxes).

Key Note:

By default, only mailbox audit events for users with licenses that include Microsoft Purview Audit (Premium) are available in audit log searches in the Microsoft Purview compliance portal or via the Office 365 Management Activity API. These licenses are described here. For brevity, this article will collectively refer to licenses that include Audit (Premium) as E5/A5/G5 licenses.

To verify that mailbox auditing on by default is turned on for your organization, run the following command in Exchange Online PowerShell:

Get-OrganizationConfig | Format-List AuditDisabled

The value False indicates that mailbox auditing on by default is turned on for the organization. Mailbox auditing on by default in the organization overrides the mailbox auditing settings on individual mailboxes.

Supported Mailbox Types

Mailbox types that are supported by mailbox auditing on by default are described in the following table:

Mailbox type	Supported
User mailboxes	✔
Shared mailboxes	✔
Microsoft 365 Group mailboxes	✔
Resource mailboxes
Public folder mailboxes

Sign-in Types Information

• Owner: The mailbox owner (the account that's associated with the mailbox).

• Delegate:

  • A user who's been assigned the SendAs, SendOnBehalf, or FullAccess permission to another mailbox.

  • An admin who's been assigned the FullAccess permission to a user's mailbox.

• Admin:

  • The mailbox is searched with one of the following Microsoft eDiscovery tools:

    • Content Search in the compliance portal.

    • eDiscovery or eDiscovery (Premium) in the compliance portal.

    • In-Place eDiscovery in Exchange Online.

  • The mailbox is accessed by using the Microsoft Exchange Server MAPI Editor.

Mailbox Actions

Mailbox action	Description
Create	An item was created in the Calendar, Contacts, Draft, Notes, or Tasks folder in the mailbox (for example, a new meeting request is created). Creating, sending, or receiving a message isn't audited. Also, creating a mailbox folder isn't audited.
FolderBind	A mailbox folder was accessed. This action is also logged when the admin or delegate opens the mailbox. (24 hour delay)
HardDelete	A message was purged from the Recoverable Items folder.
MailboxLogin	The user signed into their mailbox. (owner only login)
MailItemsAccessed	Note: This value is available only for users with E5/A5/G5 licenses. For more information, see Set up Microsoft Purview Audit (Premium). Occurs when mail data is accessed by mail protocols and clients.
MessageBind	Note: This value is available only for users without E5/A5/G5 licenses. A message was viewed in the preview pane or opened by an admin. (admin only activity, not delegate or owner)
Move	A message was moved to another folder.
MoveToDeletedItems	A message was deleted and moved to the Deleted Items folder.
RecordDelete	An item that's labeled as a record was soft-deleted (moved to the Recoverable Items folder). Items labeled as records can't be permanently deleted (purged from the Recoverable Items folder).
SearchQueryInitiated	Note: This value is available only for users with E5/A5/G5 licenses. For more information, see Set up Microsoft Purview Audit (Premium). A person uses Outlook (Windows, Mac, iOS, Android, or Outlook on the web) or the Mail app for Windows 10 to search for items in a mailbox.
Send	Note: This value is available only for users with E5/A5/G5 licenses. For more information, see Set up Microsoft Purview Audit (Premium). The user sends an email message, replies to an email message, or forwards an email message. (Owner or Admin only not delegate)
SendAs	A message was sent using the SendAs permission. This permission allows another user to send the message as though it came from the mailbox owner. (Admin or Delegate, Owner n/a)
SendOnBehalf - (admin and delegate only)	A message was sent using the SendOnBehalf permission. This permission allows another user to send the message on behalf of the mailbox owner. The message indicates to the recipient who the message was sent on behalf of and who actually sent the message.
SoftDelete	A message was permanently deleted or deleted from the Deleted Items folder. Soft-deleted items are moved to the Recoverable Items folder.
Update	A message or any of its properties was changed.
UpdateCalendarDelegation	A calendar delegation was assigned to a mailbox. Calendar delegation gives someone else in the same organization permissions to manage the mailbox owner's calendar.
UpdateComplianceTag	A retention label was updated.
UpdateFolderPermissions	A folder permission was changed. Folder permissions control which users in your organization can access folders in a mailbox and the messages located in those folders.
UpdateInboxRules	An inbox rule was added, removed, or changed. Inbox rules process messages in the user's Inbox based on conditions. Actions specify what to do to messages that match the conditions of the rule. For example, move the message to a specified folder or delete the message.

Differences between SendAs and SendOnBehalf

SendAs - Recipient does not know who actually sent the message, appears to be from impersonated mailbox

SendOnBehalf - Recipient sees the sender, and who actually sent the message.

Important:

If you customized the mailbox actions to audit before mailbox auditing on by default was turned on in your organization, the customized mailbox auditing settings are preserved on the mailbox and aren't overwritten by the default mailbox actions as described in this section. To revert the audit mailbox actions to their default values (which you can do at any time), see the Restore the default mailbox actions section later in this article.