AWS CloudTrail

Understanding AWS CloudTrail Configuration

While there are essentially two ways to forward CloudTrail logs to FortiSIEM, the FortiSIEM CloudTrail integration is designed for only one method.

Event Path

CloudTrail Generates Events -> Publish to SNS Topic -> SQS Queue is Subscribed to Topic -> FortiSIEM polls message queue (Follow this guide in its entirety, see Configuration.)

When FortiSIEM gets a message from the queue, it will be in this format (if sent direct from CloudTrail service).

{
  "Type" : "Notification",
  "MessageId" : "658da72b-684e-5965-bc3a-f123456005a",
  "TopicArn" : "arn:aws:sns:us-east-1:111111:testCloudTrail",
  "Message" : "{\"s3Bucket\":\"testS3Bucket\",\"s3ObjectKey\":[\"AWSLogs/111111/CloudTrail/us-west-2/2022/05/27/111111_CloudTrail_us-west-2_20220527T1540Z_ILkwe2zAtDS.json.gz\"]}",
  "Timestamp" : "2022-05-27T16:35:35.746Z",
  "SignatureVersion" : "1",
  "Signature" : "hS79T2w30bfMRBIUt3qJ8D0v4fAq912345I7IHzTulkme+iEjg+mWgiQV3cikFXSAwzfYVUMGwpXx+Qr7m16uW5SRIkMRb05L/5ioNrhm+DcRwjsmAEUm3ZzIFrFMaFeSy0hGD/vJEcPmvcs3ExVbz1NL1ZQcBU3LHMkrnwKKi6xFubkJWAj8nPZPUPFio7iqEHWUGHdvjqDVPkX+M7Kpwshze5q2cF6W7oPeXsUjTaV+iqFxlxi7P7TZRXsRw502wVSUYl8uVSsMKB3JdEkAJaEm3Ro/wcwxl8gbuWGwrFYwrXQoipJqv4xtrAp1ebIk/wcfMJur3mfJQ8A==",
  "SigningCertURL" : "https://sns.us-east-1.amazonaws.com/SimpleNotificationService-7ff5318522adbaddaa2a969abfda.pem",
  "UnsubscribeURL" : "https://sns.us-east-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-1:dwfewefa12323:testCloudTrail:2947799d-3c02-4863-8cd6-36123523fd1"
}

FortiSIEM parses the s3ObjectKey and retrieves the given file from the configured S3 bucket, and processes the log message.

Unsupported Message Event Path for Cloud Trail Integration

There is another way to get a notification when a log object is written to an S3 bucket, using "S3 Bucket Event Notification". This flow involves configuring an S3 bucket to publish a message event to an SNS topic when a file is placed in the bucket (or some other operation). This message is similar to the events direct from CloudTrail, but the message format is different.

Event Path for Unsupported Method

S3 Bucket creates Event notification on ObjectCreated:PUT -> S3 Publishes to SNS Topic -> SQS Queue is Subscribed to Topic.

The format of this message is not understood.

Our FortiSIEM integration for Cloud Trail does not support S3 Bucket Event Notifications

FAQ

Why doesn't FortiSIEM support this method for CloudTrail logging?

For the FortiSIEM CloudTrail integration, FortiSIEM expects an SQS queue dedicated to CloudTrail message ingest. Using the CloudTrail service to publish to SNS->Queue ensures the integration only gets CloudTrail logs to process. Using S3 event notification will send messages for non-CloudTrail objects and cause an error.

Generic AWS S3 Log Ingestion for Anything other than CloudTrail

Starting in 6.5.0, for generic log ingestion via S3 bucket event notifications, see Amazon Simple Storage Service (AWS S3).

AWS CloudTrail Topics

• What is Discovered and Monitored
• Event Types
• Reports
• Configuration
• Configure Simple Queue Service (SQS) Delivery
• Set Up Simple Notification Service (SNS)
• Give Permission for Amazon SNS to Send Messages to SQS
• Settings for Access Credentials
• Sample Events for AWS CloudTrail
• Performance Tuning for High EPS CloudTrail Events

What is Discovered and Monitored

Protocol	Information Discovered	Metrics Collected	Used For
CloudTrail API	None	None	Security Monitoring

Event Types

In ADMIN > Device Support > Event Types, search for "Cloudtrail" to see the event types associated with this device. See the Amazon API reference for more information about the event types available for CloudTrail monitoring.

Reports

In RESOURCES > Reports, search for "cloudtrail" in the main content panel Search... field to see the rules associated with this device.

Configuration

If you have not already configured Access Keys and permissions in AWS, please follow the steps outlined in AWS Access Key IAM Permissions and IAM Policies.

FortiSIEM receives information about AWS events through the CloudTrail API. After creating an S3 bucket for the storage of log files on AWS, you then configure the Simple Notification Service (SNS) and Simple Queue Service (SQS) to create a notification for the log file and have it delivered by SQS. In your FortiSIEM virtual appliance you then enter access credentials so FortiSIEM can communicate with CloudTrail as it would any other device.

Note: Do not add any extra SNS notifications in the SQS queue. The queue should only have one SNS subscription, otherwise pulling logs will not function.

Create a New CloudTrail

1. Log in to https://console.aws.amazon.com/cloudtrail.
2. Switch to the region for which you want to generate cloud trail logs.
3. Click Trails.
4. Click on Add New Trail.
5. Enter a Trail name such as aocloudtrail.
6. Select Yes for Apply Trail to all regions.
   FortiSIEM can pull trails from all regions via a single credential.
7. Select Yes for Create a new S3 bucket.
8. For S3 bucket, enter a name like s3aocloudtrail.
9. Click Advanced.
10. Select Yes for Create a new SNS topic.
    Note: This step is required for proper configuration. Do not skip.
    
    
11. For SNS topic, enter a name like snsaocloudtrail.
12. Leave the rest of advanced settings to the default values.
13. Click Create.
    A dialog will confirm that logging is turned on.

Configure Simple Queue Service (SQS) Delivery

1. Log in to https://console.aws.amazon.com/sqs.
2. Switch to the region in which you created a new cloudtrail above.
3. Click Create New Queue.
4. Enter a Queue Name such as sqsaocloudtrail.
   

   Setting	Value
   Default Visibility Timeout	0 seconds
   Message Retention Period This must be set for between 5 and 50 minutes. A lower value is recommended for high event rates to avoid event loss.	10 minutes
   Maximum Message Size	256 KB
   Delivery Delay	0 seconds
   Receive Message Wait Time	5 seconds

5. Click Create Queue.
6. When the queue is created, click the Details tab and make note of the ARN (Amazon Resource Name), as you will need this when configuring the Simple Notification Service below and when configuring the access credentials for FortiSIEM.

Set Up Simple Notification Service (SNS)

1. Log in to https://console.aws.amazon.com/sns.
2. Switch to the region where you created the trail and SQS.
3. Select Topics.
4. Select the SNS topic snsaocloudtrail that you specified when creating a cloudtrail.
5. Click Actions > Subscribe to topic from the menu to launch the popup Create Subscription.
6. For Protocol, select Amazon SQS.
7. For Endpoint, enter the ARN of the queue that you created when setting up SQS.
8. Click Create Subscription.

Give Permission for Amazon SNS to Send Messages to SQS

1. Log in to https://console.aws.amazon.com/sqs.
2. Select the queue you created, sqsaocloudtrail.
3. In the Queue Actions menu, select Subscribe Queue to SNS Topic.
4. From the Choose a Topic dropdown, select the SNS topic snsaocloudtrail that you created earlier.
5. The Topic ARN will be automatically filled.
6. Click Subscribe.

Note: Ensure that SQS, SNS, S3 bucket and CloudTrail are in the same region.

You do not need to initiate discovery of AWS Cloud Trail, but should check that FortiSIEM is pulling events for AWS by checking for an amazon.com entry in ADMIN > Setup > Pull Events.

You can configure FortiSIEM to communicate with your device by following the instructions in "Discovery Settings" and "Setting Credentials" in the User Guide.

Settings for Access Credentials

Use these Access Method Definition settings to allow FortiSIEM to communicate with the CloudTrail API.

Setting	Value
Name	aocloudtrail
Device Type	Amazon AWS CloudTrail
Access Protocol	Amazon AWS CloudTrail
Region	Region where you created the trail.
Bucket	The name of the S3 bucket you created (s3aocloudtrail)
SQS Queue URL	Enter the ARN of your queue without the http:// prefix.
Password Config	See Password Configuration.
Access Key ID	The access key for your AWS instance.
Secret Key	The secret key for your AWS instance.
Organization	Select an organization from the drop-down list.

Sample Events for AWS CloudTrail

Fri Oct 10 14:44:23 2014 FortiSIEM-CloudTrail [additionalEventData/LoginTo]=https://console.aws.amazon.com/console/home?state= hashArgs%23&isauthcode=true

[additionalEventData/MFAUsed]=No [additionalEventData/MobileVersion]=No [awsRegion]=us-east-1 [eventID]=fdf8f837-7e75-46a0-ac95-b6d15993ebf7 [eventName]=ConsoleLogin [eventSource]=SIGNIN [eventTime]=2014-10-10T06:38:11Z [eventVersion]=1.01 [requestParameters]=null [responseElements/ConsoleLogin]=Success [sourceIPAddress]=211.144.207.10 [userAgent]=Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36 [userIdentity/accountId]=623885071509 [userIdentity/arn]=arn:aws:iam::623885071509:user/John.Doe [userIdentity/principalId]=AIDAIUSNMEIUYBS7AN4UW [userIdentity/type]=IAMUser [userIdentity/userName]=John.Doe

Fri Oct 10 14:19:45 2014 FortiSIEM-CloudTrail [awsRegion]=us-east-1 [eventID]=351bda80-39d4-41ed-9e4d-86d6470c2436 [eventName]=DescribeInstances [eventSource]=EC2 [eventTime]=2014-10-10T06:12:24Z [eventVersion]=1.01 [requestID]=2d835ae2-176d-4ea2-8523-b1a09585e803 [requestParameters/filterSet/items/0/name]=private-ip-address [requestParameters/filterSet/items/0/valueSet/items/0/value]=10.0.0.233 [responseElements]=null [sourceIPAddress]=211.144.207.10 [userAgent]=aws-sdk-php2/2.4.7 Guzzle/3.7.1 curl/7.19.7 PHP/5.3.3 [userIdentity/accessKeyId]=AKIAI2MUUCROHFSLLT3A [userIdentity/accountId]=623885071509 [userIdentity/arn]=arn:aws:iam::623885071509:root [userIdentity/principalId]=623885071509 [userIdentity/type]=Root [userIdentity/userName]=fsiem

Performance Tuning for High EPS CloudTrail Events

AWS CloudTrail can generate a lot of events. Follow these recommendations to enable FortiSIEM to keep up with high EPS CloudTrail events.

1. In the AWS configuration, change the Message retention period of SQS to 1 day.
2. Adjust the CloudTrail event pulling parameters as follows. Go to the Collector that pulls AWS CloudTrail events. You will find these three relevant parameters in the /opt/phoenix/config/phoenix_config.txt file:
   • cloudtrail_msg_pull_interval (default 30 seconds, minimum recommended 10 seconds) - how often CloudTrail events are pulled.
   • cloudtrail_msg_pull_thread_num (default 1, maximum recommended 60) - how many threads are used to pull CloudTrail events.
   • cloudtrail_file_parse_thread_num (default 3, maximum recommended 60) - how many threads are used to parse CloudTrail events.

Since each API call returns maximum 10 files, set the parameters to satisfy the following two constraints. If the thread count is high, then you must increase the number of vCPUs in the Collector.

• Set (SQSInputEventRate times cloudtrail_msg_pull_interval) to be smaller than (cloudtrail_msg_pull_thread_num times 10)
• Set cloudtrail_msg_pull_thread_num to be equal to cloudtrail_file_parse_thread_num