Fortinet black logo

External Systems Configuration Guide

AWS Kinesis

AWS Kinesis

Amazon Kinesis is an Amazon Web Service (AWS) for processing big data in real time. Kinesis is capable of processing hundreds of terabytes per hour from high volumes of streaming data from sources such as operating logs, financial transactions and social media feeds.

What is Discovered and Monitored

Protocol Information collected Used for
Amazon AWS Client Library Streaming data Collect, process, and analyze real-time streaming data.

Event Types

In RESOURCES > Event Types, enter "Kinesis" in the main content panel Search... field to see the event types associated with this device.

Rules

No defined rules.

Reports

No defined reports.

Configuring AWS Kinesis

  1. Use your AWS account ID or account alias, your IAM user name, and your password to sign in to the IAM console.
  2. In the navigation bar on the upper right, choose your user name, and then choose My Security Credentials.
  3. On the AWS IAM Credentials tab, in the Access keys for CLI, SDK, and API access section, do any of the following:
    • To create an access key, choose Create access key. Then choose Download .csv file to save the access key ID and secret access key to a .csv file on your computer. Store the file in a secure location. You will not have access to the secret access key again after this dialog box closes. After you have downloaded the .csv file, choose Close. When you create an access key, the key pair is active by default, and you can use the pair right away.
    • To disable an active access key, choose Make inactive.
    • To reenable an inactive access key, choose Make active.
    • To delete an access key, choose its X button at the far right of the row. Then choose Delete to confirm. When you delete an access key, it's gone forever and cannot be retrieved. However, you can always create new keys.

Configuring FortiSIEM

  1. Go to the ADMIN > Setup > Credentials tab.
  2. In Step 1: Enter Credentials:
    1. Follow the instructions in “Setting Credentials“ in the User's Guide to create a new credential.
    2. Enter these settings in the Access Method Definition dialog box and click Save:

      Settings Description
      Name Enter a name for the credential
      Device Type Amazon AWS Kinesis
      Access Protocol AWS Kinesis Client Library
      RegionYou can enter one or more regions separated by a space, for example, “us-east-1 us-west-2”. See Supported Regions in AWS for a list of valid regions.
      Password ConfigChoose Manual, CyberArk SDK, CyberArk REST API, or RAX_Janus from the drop down list. For CyberArk SDK, see CyberArk SDK Password Configuration. For CyberArk REST API, see CyberArk REST API Password Configuration. For RAX_Janus, see RAX_Janus Password Configuration.
      Access KeyAccess key for your AWS Kinesis instance. See Configuring AWS Kinesis.
      Secret Key Secret key for your AWS Security Hub instance
      Organization The organization the device belongs to.
      Description Description of the device.
  3. In Step 2: Enter IP Range to Credential Associations, click New.
    1. Enter a host name, an IP, or an IP range in the IP/Host Name field.
    2. Select the name of your AWS Kinesis credential from the Credentials drop-down list.
    3. Click Save.
  4. Click the Test drop-down list and select Test Connectivity to test the connection to AWS Kinesis.
  5. To see the jobs associated with AWS Kinesis, select ADMIN > Setup > Pull Events.
  6. To see the received events select ANALYTICS, then enter "AWS Kinesis" in the search box.

Configuring AWS CloudTrail Logs through Kinesis Streams

The data flow for this setup is AWS CloudTrail -> AWS CloudWatch -> AWS Kinesis Data Streams.

Taking the following steps to configure.

Prerequisite
  • Obtain your access key and secret for a user account with administrator privileges.

  • Ensure the AWS CLI is installed. Fore more information on AWS CLI, see here.

  1. Run from cmd:
    aws configure

    Enter your access key, secret and default region.

    Example:

    Account ID: 11111111111

    Access Key ID: aaaaaaaaaaaaaaaaaaaaa

    Access Secret: 13452322222222222222

    Region: us-west-1

    Kinesis Role: SIEMKinesisRole

    Note: This will be created in a later portion of this guide.

  2. Create log group by entering the following:

    aws logs create-log-group --log-group-name "cloudwatch-group"

    aws logs describe-log-groups

  3. Create a cloudtrail role that permits sending logs to the CloudWatch log group by putting the following JSON into a file, and save it as a JSON file on your desktop.

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "",
          "Effect": "Allow",
          "Principal": {
            "Service": "cloudtrail.amazonaws.com"
          },
          "Action": "sts:AssumeRole"
        }
      ]
    }
    
  4. Use the AWS CLI to run the following command, referencing the file path where you saved the JSON file. In this command example, the file above was saved as cloudtrail-role.json.

    aws iam create-role --role-name CTRole --assume-role-policy-document file://cloudtrail-role.json

  5. Create the CloudTrail policy-document by saving the following JSON into a file, saving it to your desktop.

    {
      "Version": "2012-10-17",
      "Statement": [
        {
    
          "Sid": "AWSCloudTrailCreateLogStream2014110",
          "Effect": "Allow",
          "Action": [
            "logs:CreateLogStream"
          ],
          "Resource": [
            "arn:aws:logs:us-west-1:11111111111:log-group:CloudTrail:log-stream:11111111111_CloudTrail_us-west-1*"
          ]
    
        },
        {
          "Sid": "AWSCloudTrailPutLogEvents20141101",
          "Effect": "Allow",
          "Action": [
            "logs:PutLogEvents"
          ],
          "Resource": [
            "arn:aws:logs:us-west-1:11111111111:log-group:CloudTrail:log-stream:11111111111_CloudTrail_us-west-1*"
          ]
        }
      ]
    }
    
  6. Use the AWS CLI to run the following command, referencing the file path where you saved the JSON file. In this command example, the file above was saved as cloudtrail-policy.json. Note: What we're doing here is creating a role called “CTRole” and applying a policy to it.

    aws iam put-role-policy --role-name CTRole --policy-name cloudtrail-policy --policy-document file://cloudtrail-policy.json


Kinesis Configuration Setup

Take the following steps to configure Kinesis. Note: The latest instructions can be found here.

  1. Create an AWS Kinesis stream (if it does not exist). In this example, we'll create one called "fortisiem".

    aws kinesis create-stream --stream-name "fortisiem"

  2. Create a file called "TrustPolicyForCWL.json" using the following code.

    {
      "Statement": {
        "Effect": "Allow",
        "Principal": { "Service": "logs.us-west-1.amazonaws.com" },
        "Action": "sts:AssumeRole"
      }
    }
    
  3. Create an iam role with the defined policy, called "CWLtoKinesisRole" by running the following command.

    aws iam create-role --role-name CWLtoKinesisRole --assume-role-policy-document file://~/TrustPolicyForCWL.json

  4. Create a permission file called "PermissionsForCWL.json" using the following code.

    {
      "Statement": [
        {
          "Effect": "Allow",
          "Action": "kinesis:PutRecord",
          "Resource": "arn:aws:kinesis:us-west-1: 11111111111:stream/fortisiem"
        }
      ]
    }
    
  5. Associate the above policy file with the role we created, by running the following command.

    aws iam put-role-policy --role-name CWLtoKinesisRole --policy-name Permissions-Policy-For-CWL --policy-document

  6. Create the subscription to forward cloudwatch logs.

    Note: The following multiline command is for Windows, as ^ indicates the command continues on the next line. If using Linux, use the multiline command for Linux, which replaces the ^ with \. Paste all of it in as one command.

    Windows:

    //Create  CloudWatch Log Susbscription filter:
    aws logs put-subscription-filter ^
        --log-group-name "cloudwatch-group" ^
        --filter-name "fortisiem" ^
        --filter-pattern "" ^
        --destination-arn "arn:aws:kinesis:us-west-1:11111111111:stream/fortisiem " ^
        --role-arn "arn:aws:iam::11111111111:role/CWLtoKinesisRole "

    Linux:

    //Create  CloudWatch Log Susbscription filter:
    aws logs put-subscription-filter \
        --log-group-name "cloudwatch-group" \
        --filter-name "fortisiem" \
        --filter-pattern "" \
        --destination-arn "arn:aws:kinesis:us-west-1:11111111111:stream/fortisiem " \
        --role-arn "arn:aws:iam::11111111111:role/CWLtoKinesisRole "
    
  7. Create an IAM user account that FortiSIEM can use to ingest Kinesis Data Streams. When assigning permissions for the IAM user, the following must be allowed.

    Consumer
    Actions Resource Purpose
    DescribeStream Kinesis data stream Before attempting to read records, the consumer checks if the stream exists and is active, and if the shards are contained in the stream.
    Get Records, Get ShardIterator Kinesis data stream Read records from a Kinesis Data Streams shard.
    CreateTable, DescribeTable, GetItem, PutItem, Scan, Update Item Amazon DynamoDB table If the consumer is developed using the Kinesis Client Library (KCL), it needs permissions to a DynamoDB table to track the processing state of the application. The first consumer started creates the table.
    Delete Item Amazon DynamoDB table For when the consumder performs split/merge operations on Kinesis Data Stream shards.
    PutMetricData Amazon CloudWatch log The KCL also uploads metrics to CloudWatch, which are useful for monitoring the application.


  8. Navigate to IAM -> Users -> Add user
    Access Type: Programmatic Access
    Next Permission:

  9. Under permissions, select Attach existing policies directly
    AmazonKinesisReadOnlyAccess
    AmazonDynamoDBFullAccess
    CloudWatchLogsFullAccess
    Note: If step 11 is followed, it replaces the more generic access policy set here in step 9, which would no longer be needed.

  10. Click Next -> Review.
    You should see that under Permissions summary, the following policies attached to the user.

    Type

    Name

    Managed policy CloudWatchLogsFullAccess
    Managed policy AmazonDynamoDBFullAccess
    Managed policy AmazonKinesisReadOnlyAccess
  11. Once the user is created, generate an access key and secret for this user. The following reference link demonstrates how to implement restrictive, minimum access policies to FortiSIEM (which is the consumer as referenced in the guide). - https://docs.aws.amazon.com/streams/latest/dev/tutorial-stock-data-kplkcl-iam.html
    Note: For advanced users, you can customize permissions policies to allow only access to a specific CloudWatch log group, specific Kinesis Stream, etc....

FortiSIEM Configuration Setup
  • Go to the ADMIN > Setup > Credentials tab.
  • In Step 1: Enter Credentials:
    1. Follow the instructions in “Setting Credentials“ in the User's Guide to create a new credential.
    2. Enter these settings in the Access Method Definition dialog box and click Save:
      Settings Description
      Name Enter a name for the credential, such as KinesisStreamIntegration
      Device Type Amazon AWS Kinesis
      Access Protocol AWS Kinesis Client Library
      RegionYou can enter one or more regions separated by a space, for example, “us-east-2 us-west-1”. See Supported Regions in AWS for a list of valid regions.
      Log Stream NameEnter "fortisiem" or the name of your stream.
      Password ConfigChoose Manual, CyberArk SDK, CyberArk REST API, or RAX_Janus from the drop down list. For CyberArk SDK, see CyberArk SDK Password Configuration. For CyberArk REST API, see CyberArk REST API Password Configuration. For RAX_Janus, see RAX_Janus Password Configuration.
      Access KeyAccess key for your AWS Kinesis instance. See Configuring AWS Kinesis.
      Secret KeyYour Secret key.
      Organization The organization the device belongs to.
      Description Description of the device.
    3. In Step 2: Enter IP Range to Credential Associations, to the right of the Search... field, select the desired collector that will poll CloudTrail logs from the drop-down list if you have more than one collector. If only one collector is available, the drop-down list will not be available.
    4. Click New.
      1. Select the CloudTrail credential you created earlier from the Credentials drop-down list. It should autofill IP/Host Name as destination "amazon.com". FortiSIEM will handle communication to the appropriate API.
      2. Click Save.
    5. Select the new mapping and click the Test drop-down list and select Test Connectivity without Ping to start the polling.
    6. To see the jobs associated with AWS Kinesis, select ADMIN > Setup > Pull Events.
    7. To see the received events select ANALYTICS, then enter "AWS Kinesis" in the search box.

    Configuring VPC Flow Logs through Kinesis Streams

    The data flow for this setup is AWS VPC Flow Logs -> AWS CloudWatch -> AWS Kinesis Data Streams.

    Taking the following steps to configure.

    Prerequisite
    • Obtain your access key and secret for a user account with administrator privileges.

    • Ensure the AWS CLI is installed. Fore more information on AWS CLI, see here.

    1. Run from cmd:
      aws configure

      Enter your access key, secret and default region.

      Example:

      Account ID: 11111111111

      Access Key ID: aaaaaaaaaaaaaaaaaaaaa

      Access Secret: 13452322222222222222

      Region: us-west-1

    2. Configure AWS VPC flow logs to send to CloudWatch logs by following similar steps documented here.


    Kinesis Configuration Setup

    Take the following steps to configure Kinesis. Note: The latest instructions can be found here.

    1. Create an AWS Kinesis stream (if it does not exist). In this example, we'll create one called "fortisiem".

      aws kinesis create-stream --stream-name "fortisiem"

    2. Create a file called "TrustPolicyForCWL.json" using the following code.

      {
        "Statement": {
          "Effect": "Allow",
          "Principal": { "Service": "logs.us-west-1.amazonaws.com" },
          "Action": "sts:AssumeRole"
        }
      }
      
    3. Create an iam role with the defined policy, called "CWLtoKinesisRole" by running the following command.

      aws iam create-role --role-name CWLtoKinesisRole --assume-role-policy-document file://~/TrustPolicyForCWL.json

    4. Create a permission file called "PermissionsForCWL.json" using the following code.

      {
        "Statement": [
          {
            "Effect": "Allow",
            "Action": "kinesis:PutRecord",
            "Resource": "arn:aws:kinesis:us-west-1: 11111111111:stream/fortisiem"
          }
        ]
      }
      
    5. Associate the above policy file with the role we created, by running the following command.

      aws iam put-role-policy --role-name CWLtoKinesisRole --policy-name Permissions-Policy-For-CWL --policy-document

    6. Create the subscription to forward cloudwatch logs.

      Note: The following multiline command is for Windows, as ^ indicates the command continues on the next line. If using Linux, use the multiline command for Linux, which replaces the ^ with \. Paste all of it in as one command.

      Windows:

      //Create  CloudWatch Log Susbscription filter:
      aws logs put-subscription-filter ^
          --log-group-name "cloudwatch-group" ^
          --filter-name "fortisiem" ^
          --filter-pattern "" ^
          --destination-arn "arn:aws:kinesis:us-west-1:11111111111:stream/fortisiem " ^
          --role-arn "arn:aws:iam::11111111111:role/CWLtoKinesisRole "

      Linux:

      //Create  CloudWatch Log Susbscription filter:
      aws logs put-subscription-filter \
          --log-group-name "cloudwatch-group" \
          --filter-name "fortisiem" \
          --filter-pattern "" \
          --destination-arn "arn:aws:kinesis:us-west-1:11111111111:stream/fortisiem " \
          --role-arn "arn:aws:iam::11111111111:role/CWLtoKinesisRole "
      
    7. Create an IAM user account that FortiSIEM can use to ingest Kinesis Data Streams. When assigning permissions for the IAM user, the following must be allowed.

      Consumer
      ActionsResourcePurpose
      DescribeStreamKinesis data streamBefore attempting to read records, the consumer checks if the stream exists and is active, and if the shards are contained in the stream.
      Get Records, Get ShardIteratorKinesis data streamRead records from a Kinesis Data Streams shard.
      CreateTable, DescribeTable, GetItem, PutItem, Scan, Update ItemAmazon DynamoDB tableIf the consumer is developed using the Kinesis Client Library (KCL), it needs permissions to a DynamoDB table to track the processing state of the application. The first consumer started creates the table.
      Delete ItemAmazon DynamoDB tableFor when the consumder performs split/merge operations on Kinesis Data Stream shards.
      PutMetricDataAmazon CloudWatch logThe KCL also uploads metrics to CloudWatch, which are useful for monitoring the application.
    8. Navigate to IAM -> Users -> Add user
      Access Type: Programmatic Access
      Next Permission:

    9. Under permissions, select Attach existing policies directly
      AmazonKinesisReadOnlyAccess
      AmazonDynamoDBFullAccess
      CloudWatchLogsFullAccess
      Note: If step 11 is followed, it replaces the more generic access policy set here in step 9, which would no longer be needed.

    10. Click Next -> Review.
      You should see that under Permissions summary, the following policies attached to the user.

      Type

      Name

      Managed policyCloudWatchLogsFullAccess
      Managed policyAmazonDynamoDBFullAccess
      Managed policyAmazonKinesisReadOnlyAccess
    11. Once the user is created, generate an access key and secret for this user. The following reference link demonstrates how to implement restrictive, minimum access policies to FortiSIEM (which is the consumer as referenced in the guide). - https://docs.aws.amazon.com/streams/latest/dev/tutorial-stock-data-kplkcl-iam.html
      Note: For advanced users, you can customize permissions policies to allow only access to a specific CloudWatch log group, specific Kinesis Stream, etc....

    FortiSIEM Configuration Setup
    1. Go to the ADMIN > Setup > Credentials tab.

    2. In Step 1: Enter Credentials:

      1. Follow the instructions in “Setting Credentials“ in the User's Guide to create a new credential.

      2. Enter these settings in the Access Method Definition dialog box and click Save:

        Settings Description
        Name Enter a name for the credential, such as KinesisStreamIntegration
        Device Type Amazon AWS Kinesis
        Access Protocol AWS Kinesis Client Library
        RegionYou can enter one or more regions separated by a space, for example, “us-east-2 us-west-1”. See Supported Regions in AWS for a list of valid regions.
        Log Stream NameEnter "fortisiem" or the name of your stream.
        Password ConfigChoose Manual, CyberArk SDK, CyberArk REST API, or RAX_Janus from the drop down list. For CyberArk SDK, see CyberArk SDK Password Configuration. For CyberArk REST API, see CyberArk REST API Password Configuration. For RAX_Janus, see RAX_Janus Password Configuration.
        Access KeyAccess key for your AWS Kinesis instance. See Configuring AWS Kinesis.
        Secret KeyYour Secret key.
        Organization The organization the device belongs to.
        Description Description of the device.
    3. In Step 2: Enter IP Range to Credential Associations, to the right of the Search... field, select the desired collector that will poll CloudTrail logs from the drop-down list if you have more than one collector. If only one collector is available, the drop-down list will not be available.

    4. Click New.

      1. Select the CloudTrail credential you created earlier from the Credentials drop-down list. It should autofill the IP/Host Name field as destination "amazon.com". FortiSIEM will handle communication to the appropriate API.

      2. Click Save.

    5. Click the Test drop-down list and select Test Connectivity without Ping to start the polling.

    6. To see the jobs associated with AWS Kinesis, select ADMIN > Setup > Pull Events.

    7. To see the received events select ANALYTICS, then enter "AWS Kinesis" in the search box.

  • Sample Events

    VPC Log Example
    2021-04-27T08:40:50.00Z [FSM-AWSKinesis] 2 311111777194 eni-054c755e644dcf32d 103.114.104.68 172.31.9.216 22898 22717 6 1 40 1619538050 1619538102 REJECT OK
    
    2021-04-27T09:14:45.00Z [FSM-AWSKinesis] 2 311111777194 eni-054c755e644dcf32d 172.31.9.216 222.187.239.109 22 43638 6 3 3996 1619540085 1619540142 ACCEPT OK
    
    CloudTrail Log Example
    2021-04-28T14:29:51.783Z [FSM-AWSKinesis] {"eventVersion":"1.08","userIdentity":{"type":"Root","principalId":"921045424829","arn":"arn:aws:iam::1234567890AB:root","accountId":"111115424829","accessKeyId":"ASIA1234567890ABCDEF","sessionContext":{"sessionIssuer":{},"webIdFederationData":{},"attributes":{"mfaAuthenticated":"false","creationDate":"2021-04-28T20:37:47Z"}}},"eventTime":"2021-04-28T21:13:07Z","eventSource":"health.amazonaws.com","eventName":"DescribeEventAggregates","awsRegion":"us-east-1","sourceIPAddress":"10.20.30.40","userAgent":"console.amazonaws.com","requestParameters":{"aggregateField":"eventTypeCategory","filter":{"eventStatusCodes":["open","upcoming"],"startTimes":[{"from":"Apr 21, 2021 9:13:07 PM"}]}},"responseElements":null,"requestID":"1a712381-62d1-4485-8ce5-109930945c62","eventID":"eb8c077f-321d-471a-b207-849f5089a428","readOnly":true,"eventType":"AwsApiCall","managementEvent":true,"eventCategory":"Management","recipientAccountId":"921045424829"}

    AWS Kinesis

    AWS Kinesis

    Amazon Kinesis is an Amazon Web Service (AWS) for processing big data in real time. Kinesis is capable of processing hundreds of terabytes per hour from high volumes of streaming data from sources such as operating logs, financial transactions and social media feeds.

    What is Discovered and Monitored

    Protocol Information collected Used for
    Amazon AWS Client Library Streaming data Collect, process, and analyze real-time streaming data.

    Event Types

    In RESOURCES > Event Types, enter "Kinesis" in the main content panel Search... field to see the event types associated with this device.

    Rules

    No defined rules.

    Reports

    No defined reports.

    Configuring AWS Kinesis

    1. Use your AWS account ID or account alias, your IAM user name, and your password to sign in to the IAM console.
    2. In the navigation bar on the upper right, choose your user name, and then choose My Security Credentials.
    3. On the AWS IAM Credentials tab, in the Access keys for CLI, SDK, and API access section, do any of the following:
      • To create an access key, choose Create access key. Then choose Download .csv file to save the access key ID and secret access key to a .csv file on your computer. Store the file in a secure location. You will not have access to the secret access key again after this dialog box closes. After you have downloaded the .csv file, choose Close. When you create an access key, the key pair is active by default, and you can use the pair right away.
      • To disable an active access key, choose Make inactive.
      • To reenable an inactive access key, choose Make active.
      • To delete an access key, choose its X button at the far right of the row. Then choose Delete to confirm. When you delete an access key, it's gone forever and cannot be retrieved. However, you can always create new keys.

    Configuring FortiSIEM

    1. Go to the ADMIN > Setup > Credentials tab.
    2. In Step 1: Enter Credentials:
      1. Follow the instructions in “Setting Credentials“ in the User's Guide to create a new credential.
      2. Enter these settings in the Access Method Definition dialog box and click Save:

        Settings Description
        Name Enter a name for the credential
        Device Type Amazon AWS Kinesis
        Access Protocol AWS Kinesis Client Library
        RegionYou can enter one or more regions separated by a space, for example, “us-east-1 us-west-2”. See Supported Regions in AWS for a list of valid regions.
        Password ConfigChoose Manual, CyberArk SDK, CyberArk REST API, or RAX_Janus from the drop down list. For CyberArk SDK, see CyberArk SDK Password Configuration. For CyberArk REST API, see CyberArk REST API Password Configuration. For RAX_Janus, see RAX_Janus Password Configuration.
        Access KeyAccess key for your AWS Kinesis instance. See Configuring AWS Kinesis.
        Secret Key Secret key for your AWS Security Hub instance
        Organization The organization the device belongs to.
        Description Description of the device.
    3. In Step 2: Enter IP Range to Credential Associations, click New.
      1. Enter a host name, an IP, or an IP range in the IP/Host Name field.
      2. Select the name of your AWS Kinesis credential from the Credentials drop-down list.
      3. Click Save.
    4. Click the Test drop-down list and select Test Connectivity to test the connection to AWS Kinesis.
    5. To see the jobs associated with AWS Kinesis, select ADMIN > Setup > Pull Events.
    6. To see the received events select ANALYTICS, then enter "AWS Kinesis" in the search box.

    Configuring AWS CloudTrail Logs through Kinesis Streams

    The data flow for this setup is AWS CloudTrail -> AWS CloudWatch -> AWS Kinesis Data Streams.

    Taking the following steps to configure.

    Prerequisite
    • Obtain your access key and secret for a user account with administrator privileges.

    • Ensure the AWS CLI is installed. Fore more information on AWS CLI, see here.

    1. Run from cmd:
      aws configure

      Enter your access key, secret and default region.

      Example:

      Account ID: 11111111111

      Access Key ID: aaaaaaaaaaaaaaaaaaaaa

      Access Secret: 13452322222222222222

      Region: us-west-1

      Kinesis Role: SIEMKinesisRole

      Note: This will be created in a later portion of this guide.

    2. Create log group by entering the following:

      aws logs create-log-group --log-group-name "cloudwatch-group"

      aws logs describe-log-groups

    3. Create a cloudtrail role that permits sending logs to the CloudWatch log group by putting the following JSON into a file, and save it as a JSON file on your desktop.

      {
        "Version": "2012-10-17",
        "Statement": [
          {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
              "Service": "cloudtrail.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
          }
        ]
      }
      
    4. Use the AWS CLI to run the following command, referencing the file path where you saved the JSON file. In this command example, the file above was saved as cloudtrail-role.json.

      aws iam create-role --role-name CTRole --assume-role-policy-document file://cloudtrail-role.json

    5. Create the CloudTrail policy-document by saving the following JSON into a file, saving it to your desktop.

      {
        "Version": "2012-10-17",
        "Statement": [
          {
      
            "Sid": "AWSCloudTrailCreateLogStream2014110",
            "Effect": "Allow",
            "Action": [
              "logs:CreateLogStream"
            ],
            "Resource": [
              "arn:aws:logs:us-west-1:11111111111:log-group:CloudTrail:log-stream:11111111111_CloudTrail_us-west-1*"
            ]
      
          },
          {
            "Sid": "AWSCloudTrailPutLogEvents20141101",
            "Effect": "Allow",
            "Action": [
              "logs:PutLogEvents"
            ],
            "Resource": [
              "arn:aws:logs:us-west-1:11111111111:log-group:CloudTrail:log-stream:11111111111_CloudTrail_us-west-1*"
            ]
          }
        ]
      }
      
    6. Use the AWS CLI to run the following command, referencing the file path where you saved the JSON file. In this command example, the file above was saved as cloudtrail-policy.json. Note: What we're doing here is creating a role called “CTRole” and applying a policy to it.

      aws iam put-role-policy --role-name CTRole --policy-name cloudtrail-policy --policy-document file://cloudtrail-policy.json


    Kinesis Configuration Setup

    Take the following steps to configure Kinesis. Note: The latest instructions can be found here.

    1. Create an AWS Kinesis stream (if it does not exist). In this example, we'll create one called "fortisiem".

      aws kinesis create-stream --stream-name "fortisiem"

    2. Create a file called "TrustPolicyForCWL.json" using the following code.

      {
        "Statement": {
          "Effect": "Allow",
          "Principal": { "Service": "logs.us-west-1.amazonaws.com" },
          "Action": "sts:AssumeRole"
        }
      }
      
    3. Create an iam role with the defined policy, called "CWLtoKinesisRole" by running the following command.

      aws iam create-role --role-name CWLtoKinesisRole --assume-role-policy-document file://~/TrustPolicyForCWL.json

    4. Create a permission file called "PermissionsForCWL.json" using the following code.

      {
        "Statement": [
          {
            "Effect": "Allow",
            "Action": "kinesis:PutRecord",
            "Resource": "arn:aws:kinesis:us-west-1: 11111111111:stream/fortisiem"
          }
        ]
      }
      
    5. Associate the above policy file with the role we created, by running the following command.

      aws iam put-role-policy --role-name CWLtoKinesisRole --policy-name Permissions-Policy-For-CWL --policy-document

    6. Create the subscription to forward cloudwatch logs.

      Note: The following multiline command is for Windows, as ^ indicates the command continues on the next line. If using Linux, use the multiline command for Linux, which replaces the ^ with \. Paste all of it in as one command.

      Windows:

      //Create  CloudWatch Log Susbscription filter:
      aws logs put-subscription-filter ^
          --log-group-name "cloudwatch-group" ^
          --filter-name "fortisiem" ^
          --filter-pattern "" ^
          --destination-arn "arn:aws:kinesis:us-west-1:11111111111:stream/fortisiem " ^
          --role-arn "arn:aws:iam::11111111111:role/CWLtoKinesisRole "

      Linux:

      //Create  CloudWatch Log Susbscription filter:
      aws logs put-subscription-filter \
          --log-group-name "cloudwatch-group" \
          --filter-name "fortisiem" \
          --filter-pattern "" \
          --destination-arn "arn:aws:kinesis:us-west-1:11111111111:stream/fortisiem " \
          --role-arn "arn:aws:iam::11111111111:role/CWLtoKinesisRole "
      
    7. Create an IAM user account that FortiSIEM can use to ingest Kinesis Data Streams. When assigning permissions for the IAM user, the following must be allowed.

      Consumer
      Actions Resource Purpose
      DescribeStream Kinesis data stream Before attempting to read records, the consumer checks if the stream exists and is active, and if the shards are contained in the stream.
      Get Records, Get ShardIterator Kinesis data stream Read records from a Kinesis Data Streams shard.
      CreateTable, DescribeTable, GetItem, PutItem, Scan, Update Item Amazon DynamoDB table If the consumer is developed using the Kinesis Client Library (KCL), it needs permissions to a DynamoDB table to track the processing state of the application. The first consumer started creates the table.
      Delete Item Amazon DynamoDB table For when the consumder performs split/merge operations on Kinesis Data Stream shards.
      PutMetricData Amazon CloudWatch log The KCL also uploads metrics to CloudWatch, which are useful for monitoring the application.


    8. Navigate to IAM -> Users -> Add user
      Access Type: Programmatic Access
      Next Permission:

    9. Under permissions, select Attach existing policies directly
      AmazonKinesisReadOnlyAccess
      AmazonDynamoDBFullAccess
      CloudWatchLogsFullAccess
      Note: If step 11 is followed, it replaces the more generic access policy set here in step 9, which would no longer be needed.

    10. Click Next -> Review.
      You should see that under Permissions summary, the following policies attached to the user.

      Type

      Name

      Managed policy CloudWatchLogsFullAccess
      Managed policy AmazonDynamoDBFullAccess
      Managed policy AmazonKinesisReadOnlyAccess
    11. Once the user is created, generate an access key and secret for this user. The following reference link demonstrates how to implement restrictive, minimum access policies to FortiSIEM (which is the consumer as referenced in the guide). - https://docs.aws.amazon.com/streams/latest/dev/tutorial-stock-data-kplkcl-iam.html
      Note: For advanced users, you can customize permissions policies to allow only access to a specific CloudWatch log group, specific Kinesis Stream, etc....

    FortiSIEM Configuration Setup
  • Go to the ADMIN > Setup > Credentials tab.
  • In Step 1: Enter Credentials:
    1. Follow the instructions in “Setting Credentials“ in the User's Guide to create a new credential.
    2. Enter these settings in the Access Method Definition dialog box and click Save:
      Settings Description
      Name Enter a name for the credential, such as KinesisStreamIntegration
      Device Type Amazon AWS Kinesis
      Access Protocol AWS Kinesis Client Library
      RegionYou can enter one or more regions separated by a space, for example, “us-east-2 us-west-1”. See Supported Regions in AWS for a list of valid regions.
      Log Stream NameEnter "fortisiem" or the name of your stream.
      Password ConfigChoose Manual, CyberArk SDK, CyberArk REST API, or RAX_Janus from the drop down list. For CyberArk SDK, see CyberArk SDK Password Configuration. For CyberArk REST API, see CyberArk REST API Password Configuration. For RAX_Janus, see RAX_Janus Password Configuration.
      Access KeyAccess key for your AWS Kinesis instance. See Configuring AWS Kinesis.
      Secret KeyYour Secret key.
      Organization The organization the device belongs to.
      Description Description of the device.
    3. In Step 2: Enter IP Range to Credential Associations, to the right of the Search... field, select the desired collector that will poll CloudTrail logs from the drop-down list if you have more than one collector. If only one collector is available, the drop-down list will not be available.
    4. Click New.
      1. Select the CloudTrail credential you created earlier from the Credentials drop-down list. It should autofill IP/Host Name as destination "amazon.com". FortiSIEM will handle communication to the appropriate API.
      2. Click Save.
    5. Select the new mapping and click the Test drop-down list and select Test Connectivity without Ping to start the polling.
    6. To see the jobs associated with AWS Kinesis, select ADMIN > Setup > Pull Events.
    7. To see the received events select ANALYTICS, then enter "AWS Kinesis" in the search box.

    Configuring VPC Flow Logs through Kinesis Streams

    The data flow for this setup is AWS VPC Flow Logs -> AWS CloudWatch -> AWS Kinesis Data Streams.

    Taking the following steps to configure.

    Prerequisite
    • Obtain your access key and secret for a user account with administrator privileges.

    • Ensure the AWS CLI is installed. Fore more information on AWS CLI, see here.

    1. Run from cmd:
      aws configure

      Enter your access key, secret and default region.

      Example:

      Account ID: 11111111111

      Access Key ID: aaaaaaaaaaaaaaaaaaaaa

      Access Secret: 13452322222222222222

      Region: us-west-1

    2. Configure AWS VPC flow logs to send to CloudWatch logs by following similar steps documented here.


    Kinesis Configuration Setup

    Take the following steps to configure Kinesis. Note: The latest instructions can be found here.

    1. Create an AWS Kinesis stream (if it does not exist). In this example, we'll create one called "fortisiem".

      aws kinesis create-stream --stream-name "fortisiem"

    2. Create a file called "TrustPolicyForCWL.json" using the following code.

      {
        "Statement": {
          "Effect": "Allow",
          "Principal": { "Service": "logs.us-west-1.amazonaws.com" },
          "Action": "sts:AssumeRole"
        }
      }
      
    3. Create an iam role with the defined policy, called "CWLtoKinesisRole" by running the following command.

      aws iam create-role --role-name CWLtoKinesisRole --assume-role-policy-document file://~/TrustPolicyForCWL.json

    4. Create a permission file called "PermissionsForCWL.json" using the following code.

      {
        "Statement": [
          {
            "Effect": "Allow",
            "Action": "kinesis:PutRecord",
            "Resource": "arn:aws:kinesis:us-west-1: 11111111111:stream/fortisiem"
          }
        ]
      }
      
    5. Associate the above policy file with the role we created, by running the following command.

      aws iam put-role-policy --role-name CWLtoKinesisRole --policy-name Permissions-Policy-For-CWL --policy-document

    6. Create the subscription to forward cloudwatch logs.

      Note: The following multiline command is for Windows, as ^ indicates the command continues on the next line. If using Linux, use the multiline command for Linux, which replaces the ^ with \. Paste all of it in as one command.

      Windows:

      //Create  CloudWatch Log Susbscription filter:
      aws logs put-subscription-filter ^
          --log-group-name "cloudwatch-group" ^
          --filter-name "fortisiem" ^
          --filter-pattern "" ^
          --destination-arn "arn:aws:kinesis:us-west-1:11111111111:stream/fortisiem " ^
          --role-arn "arn:aws:iam::11111111111:role/CWLtoKinesisRole "

      Linux:

      //Create  CloudWatch Log Susbscription filter:
      aws logs put-subscription-filter \
          --log-group-name "cloudwatch-group" \
          --filter-name "fortisiem" \
          --filter-pattern "" \
          --destination-arn "arn:aws:kinesis:us-west-1:11111111111:stream/fortisiem " \
          --role-arn "arn:aws:iam::11111111111:role/CWLtoKinesisRole "
      
    7. Create an IAM user account that FortiSIEM can use to ingest Kinesis Data Streams. When assigning permissions for the IAM user, the following must be allowed.

      Consumer
      ActionsResourcePurpose
      DescribeStreamKinesis data streamBefore attempting to read records, the consumer checks if the stream exists and is active, and if the shards are contained in the stream.
      Get Records, Get ShardIteratorKinesis data streamRead records from a Kinesis Data Streams shard.
      CreateTable, DescribeTable, GetItem, PutItem, Scan, Update ItemAmazon DynamoDB tableIf the consumer is developed using the Kinesis Client Library (KCL), it needs permissions to a DynamoDB table to track the processing state of the application. The first consumer started creates the table.
      Delete ItemAmazon DynamoDB tableFor when the consumder performs split/merge operations on Kinesis Data Stream shards.
      PutMetricDataAmazon CloudWatch logThe KCL also uploads metrics to CloudWatch, which are useful for monitoring the application.
    8. Navigate to IAM -> Users -> Add user
      Access Type: Programmatic Access
      Next Permission:

    9. Under permissions, select Attach existing policies directly
      AmazonKinesisReadOnlyAccess
      AmazonDynamoDBFullAccess
      CloudWatchLogsFullAccess
      Note: If step 11 is followed, it replaces the more generic access policy set here in step 9, which would no longer be needed.

    10. Click Next -> Review.
      You should see that under Permissions summary, the following policies attached to the user.

      Type

      Name

      Managed policyCloudWatchLogsFullAccess
      Managed policyAmazonDynamoDBFullAccess
      Managed policyAmazonKinesisReadOnlyAccess
    11. Once the user is created, generate an access key and secret for this user. The following reference link demonstrates how to implement restrictive, minimum access policies to FortiSIEM (which is the consumer as referenced in the guide). - https://docs.aws.amazon.com/streams/latest/dev/tutorial-stock-data-kplkcl-iam.html
      Note: For advanced users, you can customize permissions policies to allow only access to a specific CloudWatch log group, specific Kinesis Stream, etc....

    FortiSIEM Configuration Setup
    1. Go to the ADMIN > Setup > Credentials tab.

    2. In Step 1: Enter Credentials:

      1. Follow the instructions in “Setting Credentials“ in the User's Guide to create a new credential.

      2. Enter these settings in the Access Method Definition dialog box and click Save:

        Settings Description
        Name Enter a name for the credential, such as KinesisStreamIntegration
        Device Type Amazon AWS Kinesis
        Access Protocol AWS Kinesis Client Library
        RegionYou can enter one or more regions separated by a space, for example, “us-east-2 us-west-1”. See Supported Regions in AWS for a list of valid regions.
        Log Stream NameEnter "fortisiem" or the name of your stream.
        Password ConfigChoose Manual, CyberArk SDK, CyberArk REST API, or RAX_Janus from the drop down list. For CyberArk SDK, see CyberArk SDK Password Configuration. For CyberArk REST API, see CyberArk REST API Password Configuration. For RAX_Janus, see RAX_Janus Password Configuration.
        Access KeyAccess key for your AWS Kinesis instance. See Configuring AWS Kinesis.
        Secret KeyYour Secret key.
        Organization The organization the device belongs to.
        Description Description of the device.
    3. In Step 2: Enter IP Range to Credential Associations, to the right of the Search... field, select the desired collector that will poll CloudTrail logs from the drop-down list if you have more than one collector. If only one collector is available, the drop-down list will not be available.

    4. Click New.

      1. Select the CloudTrail credential you created earlier from the Credentials drop-down list. It should autofill the IP/Host Name field as destination "amazon.com". FortiSIEM will handle communication to the appropriate API.

      2. Click Save.

    5. Click the Test drop-down list and select Test Connectivity without Ping to start the polling.

    6. To see the jobs associated with AWS Kinesis, select ADMIN > Setup > Pull Events.

    7. To see the received events select ANALYTICS, then enter "AWS Kinesis" in the search box.

  • Sample Events

    VPC Log Example
    2021-04-27T08:40:50.00Z [FSM-AWSKinesis] 2 311111777194 eni-054c755e644dcf32d 103.114.104.68 172.31.9.216 22898 22717 6 1 40 1619538050 1619538102 REJECT OK
    
    2021-04-27T09:14:45.00Z [FSM-AWSKinesis] 2 311111777194 eni-054c755e644dcf32d 172.31.9.216 222.187.239.109 22 43638 6 3 3996 1619540085 1619540142 ACCEPT OK
    
    CloudTrail Log Example
    2021-04-28T14:29:51.783Z [FSM-AWSKinesis] {"eventVersion":"1.08","userIdentity":{"type":"Root","principalId":"921045424829","arn":"arn:aws:iam::1234567890AB:root","accountId":"111115424829","accessKeyId":"ASIA1234567890ABCDEF","sessionContext":{"sessionIssuer":{},"webIdFederationData":{},"attributes":{"mfaAuthenticated":"false","creationDate":"2021-04-28T20:37:47Z"}}},"eventTime":"2021-04-28T21:13:07Z","eventSource":"health.amazonaws.com","eventName":"DescribeEventAggregates","awsRegion":"us-east-1","sourceIPAddress":"10.20.30.40","userAgent":"console.amazonaws.com","requestParameters":{"aggregateField":"eventTypeCategory","filter":{"eventStatusCodes":["open","upcoming"],"startTimes":[{"from":"Apr 21, 2021 9:13:07 PM"}]}},"responseElements":null,"requestID":"1a712381-62d1-4485-8ce5-109930945c62","eventID":"eb8c077f-321d-471a-b207-849f5089a428","readOnly":true,"eventType":"AwsApiCall","managementEvent":true,"eventCategory":"Management","recipientAccountId":"921045424829"}