ArcSight v1.3.1

About the connector

ArcSight Enterprise Security Manager (ESM) is a threat detection, analysis, triage, and compliance management SIEM platform.

This document provides information about the HP ArcSight connector, which facilitates automated interactions, with an ArcSight ESM server using FortiSOAR™ playbooks. Add the HP ArcSight connector as a step in FortiSOAR™ playbooks and perform automated operations, such as annotating events, running a report based on a report ID, and uploading an ArcSight report file as an attachment in FortiSOAR™.

Version information

Connector Version: 1.3.1

FortiSOAR™ Version Tested on: 4.11.0-1161

ArcSight Version Tested on: 6.11

Authored By: Fortinet

Certified: Yes

Release Notes for version 1.3.1

Following enhancements have been made to the HP ArcSight Connector in version 1.3.1:

• Enhanced the FortiSOAR™ forwarder to ensure guaranteed delivery of messages.
• Added the following operations:
  • Get Active Lists
  • Update Active List
• Updated the sample playbook collection.
• Added output schema for all operations.
• Renamed the following actions:
  • Get Event Information renamed to Get Event Details some actions and its parameters.
  • Delete Archived Report renamed to Delete Report
  • Add Events in Case to Add Events to Case
  • Delete Events from Case to Delete Case Events
  • Run Search to Search Query
  • Upload Report file as attachment to CyOps to Upload Report
• Renamed input parameters in the following operations:
  • Run Report with Default Parameters
  • Delete Report
  • Create Case
  • Update Case
  • Upload Report

Installing the connector

All connectors provided by FortiSOAR™ are delivered using a FortiSOAR™ repository. Therefore, you must set up your FortiSOAR™ repository and use the yum command to install connectors:

yum install cyops-connector-arcsight

For the detailed procedure to install a connector, click here.

Prerequisites to configuring the connector

• You must have the IP Address or FQDN of ArcSight ESM server and credentials to access the server.
• For forwarding event details from the ArcSight ESM to FortiSOAR™, you must configure the forwarding script to run with Python 3 as a local script or as a Flex CounterACT connector. See Configuring the FortiSOAR™ Forwarder topic.

Configuring the FortiSOAR™ forwarder

The FortiSOAR™ forwarder script is built to forward events from ArcSight into FortiSOAR™ for remediation, escalation, case management, and orchestrated response. You can then track correlated events according to the information from ArcSight and enrich the information with additional data, such as affected asset context and reports. The FortiSOAR™ forwarder script is bundled in the arcsight.tgz connector bundle in the scriptsdirectory (arcsight.tgz/scripts). You can configure the script to either run as a command on the ArcSight ESM machine or as a Flex CounterACT connector on a separate machine. We recommend that you configure the FortiSOAR™ forwarder script as a Flex CounterACT connector.

The following sections specify the process of configuring the FortiSOAR™ forwarder.

Configuring the FortiSOAR™ forwarder as a script

Log on to the FortiSOAR™ UI and create an appliance with appropriate permissions, based on the actions you want to perform in the FortiSOAR™ playbook with the forwarded event data, for example, create an alert. Store the public and private keys, which you will require in step 5 of the procedure.

Perform the following steps on the host where you want to install the script:

1. Install openssl-devel.
2. Install Python.
   If you are using Python 2, ensure that the version is Python 2.7 or higher.
   Install requests python package: pip install requests.
3. From the arcsight.tgz connector bundle, copy the scripts directory to the host machine. The scripts directory contains the following files and folders:
   +events
+logs
+config.ini
+cyops_forwarder.py
+cyops_parser.py
4. Update the host_uri parameter in the config.ini file to point to your FortiSOAR™ instance. Edit the following lines in config.ini:
   host_uri = '<CyOPs IP Address>'
5. In the same scripts folder, create the public_key.txt and private_key.txt files and add the public and private keys that you have stored previously.
6. Ensure that the machine can connect to the has connectivity to the FortiSOAR™ server and can send requests to the FortiSOAR™ instance on port 443.
7. To test the connectivity of the script, in FortiSOAR™ add a playbook with an Action trigger containing the same URL as in specified in the host_uri in step 4.
   Run the script as follows: <path to python> cyops_forwarder.py -eId "12345"
   The script forwards the data to FortiSOAR™ in a JSON format with the arg name as the key and value as the value. The playbook should get triggered with the payload {“-eId”: “12345”}.
   

Configuring the FortiSOAR™ forwarder as a Flex CounterACT connector

1. On the Windows or Linux machine where you have configured the forwarder script, copy the scripts/cyops.counteract.properties file from the arcsight.tgz connector bundle. See the Configuring the FortiSOAR™ forwarder as a script topic for steps on configuring the forwarder script.
   The scripts/cyops.counteract.properties file has the following contents:
   command[0].name=cyopsforwarder
command[0].displayname=Forward data to CyOPs
command[0].parameter.count=1
command[0].parameter[0].name=eventId
command [0].parameter[0].displayname=Event Id
command[0].action=python /root/arcsight/cyops_forwarder.py -eid ${eventId}

2. Edit command[0].action to point to the python executable and the forwarder script location on your machine.
   The above example forwards only the eventId to FortiSOAR™. You can extend the parameter list to add all event data that you would like to forward. Ensure that you update the command[0].parameter.count and the corresponding parameters data accordingly. The script forwards the data to FortiSOAR™ in a JSON format with the arg name as the key and value as the value. For example, if the eventId input to the script in the above sample is ‘12345’, the script sends data to FortiSOAR™ as {“-eid”: “12345”}.

3. The Flex CounterACT connector is included in the SmartConnectors package. Install the Flex CounterACT connector on the machine.

4. Copy the cyops.counteract.properties file to {ARCSIGHT_HOME} /user/agent/flexagent.

5. The Flex CounterACT connector is currently marked as INTERNAL so to install it you might need to execute an agent setup using the runagentsetup.bat script on Windows or runagentsetup.sh script on Linux. The scripts are located in your $ARCSIGHT_HOME/current/bin directory.
   If the ArcSight Manager uses a self-signed certificate, you might need to import the certificate of the ArcSight server to the trusted certificates of the connector's JRE. See the ArcSight ESM documentation for setting up the communication to the manager.
   Note: The ESM server name that you specified during installing the connector must match the server name in the server's certificate.

6. Select the Flex CounterACT connector from the list of available connectors.

7. Enter cyops as the name of the Configuration file. The extension is added automatically.

8. Start the connector: {ARCSIGHT_HOME} /bin/arcsight agents.

9. To verify that the connector has been set up correctly, select the connector in the Navigator panel of the HPE Security ArcSight Console. Right-click and select the Send Command. Under CounterACT, select the command you want to run.
   

10. A window appears, which prompts you to enter the required parameters. Enter test values for the same and click OK.
    The output of the script execution is displayed in the command result. You can also check the connector script logs from the cyops_forwarder.log log file located in the logs directory. The logs directory is part of the scripts directory and is parallel to the cyops_forwarder.py script.

Using the FortiSOAR™ ArcSight integration

The FortiSOAR™ ArcSight integration leverages a wrapper that takes the inbound syslog data from ArcSight using the Event Action feature. Once you have appropriately configured the wrapper, you can set an Event Action on a Rule such that at the time of the Playbook trigger, the wrapper is sent the Event data, which is then signed with an HMAC signature and sent to FortiSOAR™.

Create an ArcSight Rule for the set of events you need to forward to FortiSOAR™. In the Event Actions for the Rule, add Execute Connector Command or Execute Command based on whether you have configured the forwarder as a connector or as a local script.

Once the command hits the FortiSOAR™ API, you can trigger a Playbook. By default, the forwarder is configured to send data to the https://<CyOps IP>/api/triggers/1/ArcSight URL. Therefore, you must add an API trigger to the playbook with the same URL. Or, you can also edit the forwarder to send data to another desired API trigger. The playbook then leverages the HP ArcSight connector functions to perform the desired operations on the received Event data.

Working of Guaranteed delivery of messages

1. The forwarder script cyops_forwarder.py writes the event to a <timestamp>.event file and sends a success response back to the ESM.
2. A separate script, send_to_cyops.py continuously looks for events in the events directory and forwards all new events received as a group to FortiSOAR™. This script should be configured as a service or monitored using monit or similar tools. A sample systemctl config file that is used to setup the script is shipped with the forwarder. Ensure that the service is enabled and has the permissions to run the send_to_cyops.py script using python. The script should also be run with the same user as the CounterACT forwarder to ensure it has permissions to read the events directory.
   The payload for bulk send is {‘bulkevents’: [<list of event objects>]}. Once all the events are successfully sent, the forwarder script deletes the successfully sent events event file. The sample playbook 2.1 Process ArcSight Event that is shipped with the connector provides a sample for handling the bulkevents payload.
3. The forwarder purges events older than 5 days that could not be sent. You can configure the number of days after which the forwarder purges event using the days_event_retained parameter in the config.ini file.
4. The default retry count and sleep time mentioned in the first step is configurable.
   Note: Since the forwarder uses HMAC authentication for sending the events to FortiSOAR™, the time on the forwarder instance and the FortiSOAR™ instance should be in synchronization.

Configurable parameters contained in the config.ini file

Following is a list of parameters, along with its default values, that you can configure using the config.inifile:

retry_count = 2 #default number of retries in case of failure in sending events

retry_sleep = 10 # sleep between the retries

batch_max_size = 50 # max number of events that will be sent in one batch

sleep_before_next_batch = 120 # sleep for send_to_cyops.py before it starts looking for event files again

cyops_request_timeout = 30 # request timeout for sending to CyOPs

host_uri = dev.cyops,cyber # IP/FQDN of the CyOPs instance

trigger = /api/triggers/1/ArcSight # trigger for the playbook to tbe invoked with the events payload

days_event_retained = 5 # max number of days for which events are retained in case of failure to send to CyOPs

default_algorithm = sha256

Configuring the connector

For the procedure to configure a connector, click here.

Configuration parameters

In FortiSOAR™, on the Connectors page, select the HP ArcSight connector and click Configure to configure the following parameters:

Parameter	Description
Server URL	IP Address or FQDN of the ArcSight ESM server to which you will connect and perform automated operations.
ESM Port	REST API port of the ArcSight ESM server. Defaults to 8443.
Username	Username to access the ArcSight ESM server.
Password	Password to access the ArcSight ESM server.
Verify SSL	Specifies whether the SSL certificate for the server is to be verified or not. By default, this option is set as True.
Do Not Fail Connector Function On API Error?	If you select this option, i.e., set this option as True, then connector operation will not fail even when the ArcSight REST API returns a failure.

Actions supported by the connector

The following automated operations can be included in playbooks, and you can also use the annotations to access operations from FortiSOAR™ release 4.10.0 onwards:

Function	Description	Annotation and Category
Annotate Event	Updates an ArcSight Event Stage, assigns it to a user, and adds a comment.	annotate_event Investigation
Get Event Details	Retrieves information for a specific event information from the ArcSight ESM server, based on the event ID and other input parameters you have specified.	get_event_info Investigation
Run Report with Default Parameters	Runs a report based on an ID or URI and default inputs on the ArcSight ESM server.	run_report Investigation
Run Report	Runs a report based on an ID and custom user inputs on the ArcSight ESM server.	run_report Investigation
Delete Report	Deletes an archived report from the ArcSight ESM server, based on the Resource ID you have specified.	delete_report Remediation
Create Case	Creates a case in ArcSight ESM, based on the input parameters you have specified.	create_case Investigation
Update Case	Updates an existing case in ArcSight ESM, based on the input parameters you have specified.	update_case_info Investigation
Get Case Information	Retrieves information about a case from ArcSight ESM, based on the case ID you have specified.	get_case_info Investigation
Add Events to Case	Adds the specified events to an existing case in ArcSight ESM, based on the case ID you have specified.	add_events Investigation
Delete Case Events	Deletes the specified events from an existing case from ArcSight ESM, based on the case ID you have specified.	delete_events Remediation
Search Query	Searches ArcSight ESM records based on the query you have specified.	search_query Investigation
Upload Report	Downloads a report based on an ID from ArcSight ESM and then upload that report as an attachment in the FortiSOAR™ Attachment Module.	upload_report Investigation
Get Active List Information	Retrieves information about an active list from ArcSight ESM, based on the Active List ID you have specified.	get_active_list_info Investigation
Update Active List	Adds new items to a specified on ArcSight ESM, based on the Active List ID and other input parameters you have specified.	update_active_list Investigation

operation: Annotate Event

You can annotate ArcSight Events using the ArcSight Console to update the Stage and Assignee of the event and to add comments to the event.

You can also perform similar operations using the Annotate Event function in FortiSOAR™ playbooks.

Input parameters

Parameter	Description
Event ID	The ID of the ArcSight Event that you want to annotate.
Stage	The Stage to be set for the Event. You can choose from one of the following values: Queued/Initial/Monitoring/Rule Created/Follow-Up/Final/Flagged as Similar/Closed
User	An existing ArcSight user to whom you want to assign the event. For example, admin.
Comment	The comment that you want to add to the event.

Output

The JSON output returns a Success message if the ArcSight event is annotated successful or an Errormessage containing the reason for failure.

Following image displays a sample output:

operation: Get Event Details

Input parameters

Parameter	Description
Event ID	The ID of the ArcSight Event for which you want to retrieve the events.
Replace Null Values with Empty String?	If an event field is not set, the ArcSight APIs return the following values. Use this option to replace these values with an empty string. Note that, by default, the Replace Null Values with Empty String? field is set to True. Field Type integer: Returned value in place of NULL: -2147483648 (Integer.MIN_VALUE) Field Type long: Returned value in place of NULL: -9223372036854775808 (Long.MIN_VALUE) Field Type double: Returned value in place of NULL: 5e-324 (Double.MIN_VALUE
IP Address Keys to Parse	ArcSight API returns the IP address fields in decimal format. Provide a comma separated list of field names you want to convert from decimal to IP address format. Defaults to address.
MAC Address Keys to Parse	ArcSight API returns the MAC address fields in decimal format. Provide a comma separated list of field names you want to convert from decimal to MAC address format. Defaults to macAddress,translatedAddress.

Output

The JSON output contains the details of the event, based on the specified event ID, retrieved from ArcSight ESM.

Following image displays a sample output:

operation: Run Report with Default Parameters

You can get the ID for a report (Resource ID) from the ArcSight Console, as shown in the following image:

You can get the URI for a report from the ArcSight Console. To get the URI, you must add the report name to the parent resource, as shown in the following image:

Input parameters

Parameter	Description
Run Report By	Parameter of the report based on which you want to run a report on ArcSight ESM. You can choose between Report ID or Report URI.
Report URI or Report ID	ID or URI of the ArcSight report that you want to run on ArcSight ESM.

Output

The JSON output returns the Download ID of the report. You can use this Download ID to download the report subsequently when the report is ready. You can use the Upload Report file operation to download the report and add it as an attachment to FortiSOAR™.

Following image displays a sample output:

operation: Run Report

Input parameters

Parameter	Description
Report Id	ID of the ArcSight report that you want to run.
Input parameters	Input parameters in the json format. For example, {'StartTime': '$Now - 3h', 'Report Format': '0'}. The keys are the same as seen on the ArcSight console. Note that the values for the drop-down fields are their integer positions. For example, Report Format should be specified as 0, 1, 2 etc, and not as pdf, csv, html etc.

Output

The JSON output returns the Download ID of the report. You can use this Download ID to download the report subsequently when the report is ready. You can use the Upload Report operation to download the report and add it as an attachment to FortiSOAR™.

Following image displays a sample output:

operation: Delete Report

Input parameters

Parameter	Description
Report ID	ID of the archived ArcSight report that you want to delete from ArcSight ESM.

Output

The JSON output returns a Success message if the the specified report is deleted from ArcSight ESM, or an Error message containing the reason for failure.

Following image displays a sample output:

operation: Create Case

Input parameters

Parameter	Description
Parent Group ID	Parent Group ID of the case you want to create.
Case Name	Name of the case that you want to create.
Alias (Display Name)	(Optional) Alias or Display Name of the case that you want to create.
Ticket Type	(Optional) Ticket type of the case you want to create. You can choose from the following options: INTERNAL, CLIENT, or INCIDENT.
Stage	(Optional) Stage that you want to assign to the created case. You can choose from the following options: QUEUED, INITIAL, FOLLOW_UP, FINAL, or CLOSED.
Frequency	(Optional) Frequency that you want to assign to the created case. You can choose from the following options: TEN_TO_FIFTEEN, NEVER_OR_ONCE, FIFTEEN, LESS_THAN_TEN, or MORE_THAN_FIFTEEN.
Operational Impact	(Optional) Operational Impact that you want to assign to the created case. You can choose from the following options: NO_IMPACT, NO_IMMEDIATE_IMPACT, LOW_PRIORITY_IMPACT, HIGH_PRIORITY_IMPACT, or IMMEDIATE_IMPACT.
Security Classification	(Optional) Security Classification that you want to assign to the created case. You can choose from the following options: UNCLASSIFIED, CONFIDENTIAL, SECRET, or TOP_SECRET.
Consequence Severity	(Optional) Consequence Severity that you want to assign to the created case. You can choose from the following options: NONE, INSIGNIFICANT, MARGINAL, CRITICAL, or CATASTROPHIC.
External ID	(Optional) Unique ID of the case you want to create.
Description	(Optional) Description of the case you want to create.
Deprecated	(Optional) Whether or not the created case is deprecated.
Additional attributes in json format	(Optional) Use this field to set values that are not displayed in FortiSOAR™.

Output

The JSON output contains the case ID and the details of the case created on ArcSight ESM.

Following image displays a sample output:

operation: Update Case

Input parameters

Parameter	Description
Case ID	ID of the case you want to update.
Case Name	(Optional) Updated case name, if you want to update the name of an existing case.
Alias (Display Name)	(Optional) Alias or Display Name of the case that you want to update.
Ticket Type	(Optional) Ticket type of the case you want to update. You can choose from the following options: INTERNAL, CLIENT, or INCIDENT.
Stage	(Optional) Updated stage, if you want to update the stage of an existing case. You can choose from the following options: QUEUED, INITIAL, FOLLOW_UP, FINAL, or CLOSED.
Frequency	(Optional) Updated frequency, if you want to update the frequency of an existing case. You can choose from the following options: TEN_TO_FIFTEEN, NEVER_OR_ONCE, FIFTEEN, LESS_THAN_TEN, or MORE_THAN_FIFTEEN.
Operational Impact	(Optional) Updated operational impact, if you want to update the operational impact of an existing case. You can choose from the following options: NO_IMPACT, NO_IMMEDIATE_IMPACT, LOW_PRIORITY_IMPACT, HIGH_PRIORITY_IMPACT, or IMMEDIATE_IMPACT.
Security Classification	(Optional) Updated security classification, if you want to update the security classification of an existing case. You can choose from the following options: UNCLASSIFIED, CONFIDENTIAL, SECRET, or TOP_SECRET.
Consequence Severity	(Optional) Updated consequence severity, if you want to update the consequence severity of an existing case. You can choose from the following options: NONE, INSIGNIFICANT, MARGINAL, CRITICAL, or CATASTROPHIC.
Estimated Restore Date Time	(Optional) Updates the Date and time for restoring the case, if required.
External ID	(Optional) Updated External ID, if you want to update the Unique ID of the case.
Description	(Optional) Updated description of the case.
Deprecated	(Optional) Updates whether or not the case is deprecated.
Notification Group IDs	(Optional) IDs of groups that should be notified when the case is updated.
Custom Fields	(Optional) Use this field to set or update values that are not displayed in FortiSOAR™.

Output

The JSON output contains the details of the case updated on ArcSight ESM.

Following image displays a sample output:

operation: Get Case Information

Input parameters

Parameter	Description
Case ID	ID of the case for which you want to retrieve the information from ArcSight ESM.

Output

The JSON output contains the details of the case, retrieved from ArcSight ESM, based on the specified case ID.

Following image displays a sample output:

operation: Add Events to Case

Input parameters

Parameter	Description
Case ID	ID of the case in which you want to add events.
Events IDs	IDs of the events that you want to add to the specified case case. You must provide the Event IDs in the list format.

Output

The JSON output returns a Success message if the events are successfully added to the specified case ID, or an Error message containing the reason for failure.

Following image displays a sample output:

operation: Delete Case Events

Input parameters

Parameter	Description
Case ID	ID of the case from which you want to delete events.
Events IDs	IDs of the events that you want to delete from the specified case. You must provide the Event IDs in the list format.

Output

The JSON output returns a Success message if the events are successfully deleted from the specified case ID, or an Error message containing the reason for failure.

Following image displays a sample output:

operation: Search Query

Input parameters

Parameter	Description
Query	Query based on which you want to search ArcSight ESM.
Start Position	Position from where you want to start the search. By default, this is set to 0.
Page Size	Number of result records that you want to display on one page. By default, this is set to 10.

Output

The JSON output contains the search results retrieved from ArcSight ESM, based on the specified query.

Following image displays a sample output:

operation: Upload Report

Input parameters

Parameter	Description
Report ID	Download ID of the ArcSight report that you want to upload as an attachment in FortiSOAR™. Note: You can get the ID of the report using the Run Report function.
Name of the file when added as an attachment in Cybersponse	Name of the file when it is added as an attachment in FortiSOAR™. If you do not specify any name then the file by default, is named as 'ArcSight Report '.

Output

The JSON output contains the details of the attachment in FortiSOAR™.

Following image displays a sample output:

operation: Get Active List Information

Input parameters

Parameter	Description
Active List ID	Resource ID of the Active List for which you want to retrieve details from ArcSight ESM.

Output

The JSON output contains the details of the active list, retrieved from ArcSight ESM, based on the specified active list ID.

Following image displays a sample output:

operation: Update Active List

Input parameters

Parameter	Description
Active List ID	Resource ID of the Active List that you want to update on ArcSight ESM.
Column Names List	(Optional) List of column names that you want to update, i.e. columns in which you want to add entries. By default, all the column names are included.
Entry List Values	List of entries to add to the specified active list. You must add the values in the same sequence as the columns specified. For example, [[“val1”, “val2”], [“val3”, “val4”]]

Output

The JSON output returns a Success message if the active list is successfully updated on ArcSight ESM, or an Error message containing the reason for failure.

Following image displays a sample output:

Included playbooks

The Sample - HP ArcSight-1.3.1 playbook collection comes bundled with the HP ArcSight connector. This playbook contains steps using which you can perform all supported actions. You can see the bundled playbooks in the Automation > Playbooks section in FortiSOAR™ after importing the HP ArcSight connector.

• Active List : Get Active List Information
• Active List : Update Active List
• Case : Add Events to Case
• Case : Create Case
• Case : Delete Case Events
• Case : Get Case Information
• Case : Update Case
• Event : Annotate Event
• Event : Get Events
  Child playbook: Event : Create Alert
• Report : Delete Report
• Report : Run Report
• Report : Run Report with Default Parameters
• Report : Upload Report
• Search Query

Note: If you are planning to use any of the sample playbooks in your environment, ensure that you clone those playbooks and move them to a different collection since the sample playbook collection gets deleted during connector upgrade and delete.

Troubleshooting

Forwarder script is not able to send data to the FortiSOAR™ instance with authentication failure

This generally occurs when the system date-time on the host on which the forwarder is running does not match closely enough to the FortiSOAR™ system time.

Resolution:

You can first check the system clock on the host and see how closely it matches the FortiSOAR™ clock. You can then solve this issue by installing or updating the NTP on the host if it the host does not have a NPT server. Using the same NTP server for both the host and the FortiSOAR™ system ensures that the time always matches.

This error will most likely not occur once the NTP server is the same and the clocks are in sync.

Connection refusal while requesting to run the wrapper

This generally occurs in the case of self-signed SSL certificates. If you are using self-signed certificates for testing or staging, keep in mind this problem will not occur in production and you might need to switch the certificates on or off.

Resolution:

Ensure that the SSL certificates are trusted or that SSL checking is turned off in the wrapper script. This is not advised for production instances.

Playbook fails after the ingestion is triggered

There are many reasons for a playbook failure, for example, if a required field is null in the target module record, or there are problems with the Playbook Appliance keys.

Resolution:

Investigate the reason for failure using the Running Playbooks tab in the Playbook Administration page. Review the step in which the failure is being generated and the result of the step, which should contain the trace of the error. Once you have identified the error and if you cannot troubleshoot the error, contact CyberSponse support for further assistance.