Building your own connector

You can write a custom connector that can retrieve data from custom sources. You can then use the connector either standalone or within FortiSOAR playbooks and perform automated operations. From version 7.0.0, you can create your own connector using the Connector Wizard. You can also write a custom connector manually, or you can use the FortiSOAR Connector SDK to develop your custom connector.

The process of installing, configuring, and using FortiSOAR-provided connectors is defined in the respective connector-specific documentation.

Permissions required for building a connector

• To create and publish a custom connector, you must be assigned a role that has a minimum of Create, Read and Update access on the Connectors module, and Read access on the Application module.
• To delete a custom connector, you must be assigned a role that has a minimum of Create, Read, Update, and Delete access on the Connectors module, and Read access on the Application module.

Building a connector using the Connector Wizard

From version 7.0.0, FortiSOAR provides you with a connector wizard using which you can create a new connector. You can also edit an existing connector according to your requirements by clicking on the Edit Connector icon on a connector card.

To create a new connector, do the following:

1. Log on to FortiSOAR.
2. On the left navigation pane, click Automation > Connectors > Installed.
3. To create a new connector, click + > Create New Connector;
   
   This opens the "Connector Wizard":
   You can also open the Connector Wizard by clicking the Create Connector button in the Workspace tab.
   Click the Lets's start by defining a connector button to start building your connector.
   
4. In the Connector Wizard, in the About Connector screen, you need to provide the basic details such as name, description, etc. of the connector that you want to create, as well as, upload a logo for the connector, and choose a suitable template for the connector:
   
   You need to add the following details on this screen:
   1. Click the Upload icons buttons to upload large, medium, or small logos for your connector. Clicking the upload button opens the Upload Connector Logo dialog, where you can drag-and-drop your logo, or browse to the logo on your disk and import the logo.
   2. In the Connector Name field, add the name of the connector that you want to create.
      It is a good practice to include both the name of the organization and the name of the product in the connector name, for example, Fortinet FortiSOAR.
      
   3. In the API Identifier field, enter a name that would be used to as a variable in the connector code to reference this connector. The variable that you use here can be alphanumeric; however, it should not contain any special characters and it must not start with a number.
      Also, note that the value that you enter in this field must not match the name of any other connector that is available in the connector store. For example, you cannot enter 'virustotal' in this field, since the virustotal connector is available in the connector store.
      Note: You can change the connector name and API identifier for a connector by editing the info.json of that connector.
   4. In the Connector Version field, enter the version of the connector in the x.y.z format. For example, 1.0.0.
      Note: It is recommended that you increase the version number if you are making any changes to an installed connector.
   5. In the Description field, enter information for the connector that you are creating, which enables users to understand more about the connector.
   6. From the Select Template drop-down list, select the template that you want to apply to the connector that you want to create.
      Templates are based on the broader category of connector, for example, threat intelligence connectors, SIEM connectors, etc. Templates automatically add suggestive actions, configuration details, and other connector details based on the type of connector you want to create, which helps to speed up the development process.
   7. Once you have completed entering all the details, click Continue, to go to Connector Configuration screen.
5. Use the Connector Configuration screen to setup the connector configuration page. If you have specified a template in the About Connector screen, then you will see that some suggestive configuration fields are added based on the selected template, else you will not see any configuration fields and you require to add all the required configuration fields.
   If, for example, in the About Connector screen, the Threat Intel Connector is selected from the Select Template drop-down list, then you will see the following Connector Configuration screen:
   
   To add fields to the connector configuration, click the Add Field link and can easily add properties to that field, like selecting the type of field you want to add such as Text, Integer, Email, etc., and even a sub-type of Text fields such as URL, Filehash, etc., all of which help you in ensuring that the user enters appropriate values in the field. You can also set other properties such as visibility and default value of the field, whether its required or not, etc.
   Important: You cannot add fields that are named 'name' and 'default' to the connector configuration since, 'name' and 'default' are reserved keywords.
   
   To edit existing fields and change their properties, select the field to change the properties and click Apply.
6. Once you have completed editing the connector configuration and to create the connector, click Create Connector.
   Once you click Create Connector, the code editor interface for the selected connector gets displayed for you to edit and test the connector. The created connector also gets saved in the Workspace tab, from where you also edit the connector. For more information, see the Editing a connector section.
   
   

Editing a connector

You can use the connector's code editor interface to edit existing connectors and build new connectors for custom use cases.

To edit an existing connector that is present in the Connector Store and which you have installed on your system, click the Edit Connector icon on the Installed Connectors page. However, before you can make any changes to a connector installed from the connector store, you are required to create a local copy of that connector. For example, click the Edit Connector icon on the VirusTotal connector. This displays a Confirm dialog that requiring you to clone the connector before making any changes, click Confirm to open the Clone dialog, where you should update the connector name, version, and also ensure that you have specified a unique API Identifier and then click Create:

Clicking Create opens the code editor interface where you can make the required changes to the connector.

It also saves the cloned connector in the Workspace tab, with the name that is specified in the API Identifier field, i.e., 'VirusTotal v1.1.0_dev'.

You can change the connector name and API identifier for a connector by editing the info.json of that connector.

To edit the connector, click the Edit button:

If you want to edit a custom connector that has been created using the Connector Wizard and not from the Connector Store, and which has been published (installed), you are not required to clone the connector. Instead you can choose to either edit, save, and publish the same version of the connector, or choose to add new version to the connector. For example, if you click the Edit Connector icon on the 'Demo' connector (created using the Connector Wizard), the Edit Connector dialog displays the Edit and the Add Versions options:

If a connector is published and is being used by other users, it is recommended that you add a version to the connector before editing the connector code, so that the changes you make do not cause any issues to existing operations done using the connector.

Similarly, once you create a connector using the "Connector Wizard", that connector is saved in the Workspace tab, where you can edit the connector.

To edit a connector, do the following:

1. Open the Workspace tab:
   
   In the Workspace tab, you will see connector cards with their details such as the connector version, the created and the last modified dates, whether the connector is installed or not, etc.
   You might see more than one version of the connector such as the Demo connector in the above image, if you are working on multiple versions of the connector or have chosen to maintain a working copy of an installed connector in the Workspace tab, as is the case with Demo v1.0.0, i.e., in this case you have chosen the Publish Only option while publishing the connector.
   To add a version for the connector, click Add Version, which will again display the Clone Connector dialog.
2. To edit a connector, click Edit Connector, which displays the code editor interface:
   
   The files and folders that are present are similar to what is described in the Writing a custom connector manually section. A brief description of these files and folders follow:
   
   • The info.json file contains information about the name and version of the connector, logo image file names, the configuration parameters, the set of functions supported by the connector, and their input parameters. The name of all the fields in the info.json file must be unique.
     To add an operation (Action in a connector), you must add the operation parameters in the info.json file, and this in turn creates a .py file per operation. Each operation .py file contains the default template format for the operation. Once the info.json is updated with operation it is mapped in the builtins.py.
     
   • The connector.py file extends the base connector class and implements the check_health and execute functions.
     Note: The connector.py template file provides an additional function called dev_execute that can be used during the development phase of the connector. Code changes do not always reflect without a service restart; therefore dev_execute reloads the function definition at every 'execute' call to ensure that the changes made at every save in the code get immediately reflected. You can change the execute function in connector.py as follows to call the dev_execute function instead:
     def execute(self, config, operations, params, *args, **kwargs):
return self.dev_execute(config, operation, params)
     However, note that reloading the function at every function call is slow and performance-intensive. So, once the development is complete and the connector is ready to use, you must revert the code to the original. For more information on connector.py, see the connector.py section.
     
   • The images directory contains have the connector logo files.
   • The playbook directory contains the playbook.json file, which contains playbook collection that you want to include with your connector.
   • The requirements.txt file lists the additional python libraries that are required by the connector. However, the connector dependencies are not installed by default when the connector is created. If you want to test connector that is being developed, you must manually install the connector dependencies.
   • The release_notes.md file contains information such as what's new and what's fixed, in a particular release of a connector.

   Apart from the above files, it is recommended that you maintain a file (utils.py) that contains common functions such as datetime conversion, str to list, etc., which can be used by multiple actions in the connector.

3. Edit the existing connector as required, and click Save Changes.
   The connector is saved in the "Draft" state.
   The left-pane of the code editor interface contains the name of the connector along with various icons using which you create new files and folders, and also upload or delete selected files or folders:
   
   You can also rename connector files such as the requirements.txt, release_notes.md, etc., by selecting the file row and clicking the Edit Filename icon, which displays a Rename File dialog in which you can specify the new filename and click Save.
   You can also perform the following operations on the code editor interface:
   • Code Formatting: To lint your code automatically and make the code more human-readable and error-free (programming and programming errors), select the entire code in the editor and click the Format button.
   • Full Screen: To get a better working view and make the editor go full-screen, click the Fullscreen button. To exit the full screen, press ESC.
   • Export: To export the connector as a .tgz file, click the Export button. You might want to import the exported connectors' .tgz file into another system.
   • Testing the configuration: While developing a connector, you can perform health check operation on the connector. Clicking the Test Configuration button opens a dialog containing the specified configuration parameters, where you can input the values, save the configuration, and perform the health check. If the configuration is valid, i.e., the configuration parameters and values specified are correct, then the health check will appear as "Available", else it will appear as "Disconnected".
     
     
   • Testing the actions: You can use the saved configuration for testing connector actions for their output/errors. Clicking the Test Actions button displays a list of actions for that connector. Click the action that you want to test to display a dialog containing the input parameters for that action. Enter the values for the input parameters and click Execute Action to view the output for that action.
     
     Once the connector action is executed, you can see the formatted output of the action, in a tabular format, as shown in the following image:
     
   • Deleting the Connector: To delete the connector, click the Delete Connector. button.
     
4. Once you have completed making the changes, click Publish Connector to display the Publish Connector dialog:
   
   To delete all existing versions of the connector from your system, click the Delete all existing versions checkbox.
   The publish operation has the following options:
   • Publish Only: This publishes the connector and makes it available on the Installed tab for all the users of the system and use it in playbooks or run directly on records. However, a working copy for the same is also maintained in the Workspace tab.
   • Publish and Discard: This publishes the connector, makes it available on the Installed tab for all the users of the system, and also removes this connector from the Workspace tab.

Removing images of a connector installed from the connector store

If you are editing a connector that is part of the connector store, for example 'VirusTotal' and you want to remove the images of such a connector, do the following:

1. Open the connector in the code editor interface, and click the images folder.
2. Select the image that you want to delete and then click the Delete icon, then click Delete on the confirmation dialog.
3. In the code editor interface, click info.json and edit the info.json file to remove the file references of the deleted image. For example, if you have deleted the large logo for the connector (large.png), then in the info.json file, remove its reference:
   "icon_large_name": "large.png"
   
4. To save your changes, click Save Changes.

Debugging connectors

• You can add logger statements to test your connector's code paths. Use the get_logger utility function to initialize the logger. You can import get_logger from connectors.core.connector, and then declare the logger as logger = get_logger('<connector name>'). Then you can add logger.error('<logger_message>') to the operation.py file that you want to debug.

• If your connector's configuration or action fails, you can check the connector logs. All the connector logs are written to the /var/log/cyops/cyops-integrations/connectors.log file and follow the format: %(asctime)s %(levelname)s %(name)s %(module)s %(funcName)s(): %(message)s
  The default log level is WARN, however, you can change it to INFO or DEBUG.
  For example, to change the log level to INFO:
  Open the /opt/cyops-integrations/integrations/configs/config.ini file and set connector_logger_level = 'INFO', and then restart the uwsgi service.
  

Writing a custom connector manually

Perform the following steps to write a custom connector manually:

1. Create a Connector Template Directory Structure with the required files.
2. Import the connector into FortiSOAR.
3. Check the health of the connector.
4. Add a connector operation to a playbook.

Connector Template Directory Structure

Create the following folder structure, with a folder with the connector name at the top and files within it:

connectorname folder
--+ playbooks
---+ __init__.py
---+ playbooks.json
--+ connector.py
--+ info.json
--+ images
--+ requirements.txt
--+ packages
---+ <package_name>

Python 3 is required for developing your connectors.

info.json

The info.json file contains information about the name and version of the connector, logo image file names, the configuration parameters, the set of functions supported by the connector, and their input parameters. The name of all the fields in the info.json file must be unique.

Ensure that the name of the connector in the info.json and the name of the connector folder matches exactly.

You can configure the following parameters, fields, and operations while writing connectors: description of connector and actions, tooltip, placeholder, conditional fields, backward compatibility (using visible_onchange parameter), validation using regex, label, apioperations, and grouping of fields. These parameters are explained in the Notes following the sample info.json file.

Following is an example of an info.json file:

{
 "name": "sampleConnector",
 "label": "sampleConnector",
 "description": "sampleConnector connector description",
 "publisher": "publisherName",
 "cs_approved": true/false,
 "cs_compatible": true/false,
 "version": "1.0.0",
 "category": "categoryType",
 "help_online": "link to online documentation",
 "icon_small_name":"small_icon.jpeg",
 "icon_large_name":"large_icon.jpeg",
 "configuration": {
   "fields": [
     {
       "title": "fieldname",
       "required": true/false,
       "editable": true/false,
       "visible": true/false,
       "type": "text",
       "description": "text",
       "name": "user",
       "tooltip": "text for the tooltip",
       "placeholder": "placeholder text",
       "validation":{
            "pattern":"regex pattern for validation",
            "patternError":"text for the error message if validation fails"
          }
     }]
 },
 "operations": [
   {
     "operation": "function_template",
     "title": "Sample Function",
     "description": "description of the operation",
     "category": "categoryType",     
     "annotation": "sample_annotation",
     "parameters": [{
       "title": "Sample Input1",
       "required": true/false,
       "editable": true/false,
       "visible": true/false,
       "type": "text",
       "name": "input1",
       "description": "text",
       "value": "default value1"
       "tooltip": "text",
     },
     {
       "title": "Sample Input2",
       "required": true/false,
       "editable": true/false,
       "visible": true/false,
       "type": "text",
       "name": "input2",
       "description": "text",
       "tooltip": "text",
       "value": "",
       "options": ["A", "B"],
       "onchange": {
            "A": [{
              "title": "User2",
              "required": true/false,
              "editable": true/false,
              "visible": true/false,
              "visible_onchange":false,
              "type": "text",
              "name": "user2",
              "description": "text",
			  "tooltip": "text",     
              "value": "admin1"           
            }],
            "B": [{
              "title": "User2",
              "required": true/false,
              "editable": true/false,
              "visible": true/false,
              "type": "integer",
              "name": "user2",
              "description": "text",
			  "tooltip": "text",  
              "value": 12345
            },{
              "title": "Comment1",
              "required": true/false,
              "editable": true/false,
              "visible": true/false,
              "type": "text",
              "name": "comment1",
              "description": "text",
			  "tooltip": "text",  
              "placeholder": "Add comment1",
              "value": "",
              "onchange": {
                "placeholder attribute": [{
                  "title": "placehoder attribute title",
                  "required": true/false,
                  "editable": true/false,
                  "visible": true/false,
                  "type": "checkbox",
                  "name": "placehoder attribute name",
                  "description": "text",
			      "tooltip": "text",  
                  "value": false
                }
              ]}
            }]       
       }
     }],
     "enabled": true, 
     "output_schema": {"key1": "", "key2": []}
   },
   {
            "operation": "Sample Operation 2",
            "category": "containment",
            "annotation": "sample_op",
            "title": "title of the operation",
            "description": "description of the operation",
            "is_config_required": true/false,
            "parameters": [
                {
                    "title": "title of list",
                    "type": "text",
                    "name": "sample_list",
                    "required": true/false,
                    "editable": true/false,
                    "visible": true/false,
                    "description": "description of the operation",
                    "tooltip": "text"
       },
                {

                    "type": "label",
                    "label": "label text",
                    "visible": true
       },
                {
                    "title": "title of list",
                    "type": "text",
                    "name": "sample_list",
                    "description": "description of the operation",
                    "required": true/false,
                    "editable": true/false,
                    "visible": true/false,
                    "class": "group-element",
                    "tooltip": "text"
       },
                {
                    "title": "title of list",
                    "type": "text",
                    "name": "sample_list",
                    "description": "description of the operation",
                    "required": true/false,
                    "editable": true/false,
                    "visible": true/false,
                    "class": "group-element",
                    "tooltip": "text"
       },
                {
                    "title": "title of list",
                    "type": "text",
                    "name": "sample_list",
                    "description": "description of the operation",
                    "required": true/false,
                    "editable": true/false,
                    "visible": true/false,
                    "class": "group-element last",
                    "tooltip": "text"
       }
      ],
            "enabled": true,
            "output_schema": {}
    }
 ],
 "playbooks": [
   {
   }
 ]
}

Notes:

• The version of the connector must be in the x.y.z format, for example, 1.0.0. Version must consist of valid integers, for example, "1.15.125" is a valid version.
• The output_schema defines the keys that are present in the output json on the execution of an operation. The info.json contains some common keys. However, the output json can have additional keys based on the input parameters. You can use these json keys to set the input for subsequent Playbook Steps, using the Dynamic Values. For more information, see the Dynamic Values section in "Playbooks."
• If you want to add online and/or offline documentation for your connector, add the following to your info.json:
  help_file: "name of pdf file in the connector folder"
help_online: "link to the online documentation for the connector"
• Input types supported: text, checkbox, integer, decimal, datetime, phone, email, file, richtext, json, textarea, image, select, and multiselect.
  For select and multiselect, you can provide the list of inputs using the options key.
  For example,

  {  
        "title": "Sample Field",
        "name": "sample",
        "required": "true",
        "editable": "true",
        "visible": "true",
        "type": "select",
        "options": ["A","B","C"]      
  }  
  

• category and annotation within operations: The category defines the category for the connector that you are adding, and it must be one of the following: investigation, remediation, containment and miscellaneous.
  The annotation defines the operation or function that will be performed. An annotation is unique and belongs to only one category, i.e., you must not add an annotation to multiple categories.
  If you do not define any category in the info.json file, then by default, the annotation is added to the miscellaneous category.
  The category name must contain only lower-case alphabets. The annotation name must contain only lower-case alphabets, underscores, and numbers.
  Category and annotations must always come together.
  Multiple operations within a connector can use the same annotation.
• description: You can add a description for the connector, which will be visible on the connector page in FortiSOAR and you can also add a description for the action or operation, which will be visible on the connector step page in the Playbook Designer.
• is_config_required: By default, the is_config_required key is set to true (default), which means that the connector uses the configuration specified in the connector configuration to execute an action. If you set the is_config_required key to false, then configurations are not required to execute an action and in this case, you can use dynamic fields specify the configurations.
  Note: This key is applicable at the action level, i.e., limited to a single action.
• tooltip: You can add the tooltip parameter to any field to display information about that particular field. Note that if you do not want a tooltip against any field, then you must remove the tooltip parameter from that field in the info.json. You must not pass "" or null as values to the tooltip parameter.
• placeholder: You can add the placeholder parameter for text and select fields, which will display placeholders for those fields on the connector step page in the Playbook Designer.
• conditional fields (onchange): You can add the onchange parameter to fields, which you can use to display other fields or subfields conditionally based on the user input for that field. For example:

  {
    "options": ["Now", "Yesterday"],
    "onchange": {
                  "Now": [{
                    "title": "Comment3",
                    "required": true,
                    "editable": true,
                    "visible": true,
                    "type": "text",
                    "name": "comment3",
                    "value": ""
                  }], 
                  "Yesterday": [{
                    "title": "Comment4",
                    "required": true,
                    "editable": true,
                    "visible": true,
                    "type": "text",
                    "name": "comment4",
                    "value": ""
                  }]
    }
  }
  

• visible_onchange: For backward compatibility of the connector you can use the visible_onchange parameter for conditional fields (onchange parameter).
  If you set visible_onchange to false, then this field will be hidden in FortiSOAR UI, and if you set visible_onchange to true, or if it is not present for any field, then this field is visible in FortiSOAR UI.
• validation: You can add regex validation to text and textarea fields. You can also add the error that will be displayed in case the validation fails. For example:

  {
   "validation":{
              "pattern":"\\d+",
              "patternError":"this is not a number"
            }
  }
  

• label: You can add the label field to display text on the connector UI. For example:

  {
  "type": "label",
  "label": "this is my label",
  "visible": true
  }
  

• grouping: You can group fields based on your categorization using "class": "group-element". For the last field in the group, use "class": "group-element last".
  For example:

  {
         "operation": "block_applications",
         "parameters": [                 
            {
  
                "type": "label",
                "label": "this is my label",
                "visible": true
         },
             {
                 "title": "Applications Names(List Format)",
                  "type": "text",
                  "name": "app_list",
                  "required": true,
                  "editable": true,
                  "visible": true,
                  "class": "group-element",
                  "tooltip": "Block Applications Names (List Format)"
         },                 
             {
                 "title": "Applications Names(List Format)",
                 "type": "text",
                  "name": "app_list",
                  "required": true,
                  "editable": true,
                  "visible": true,
                 "class": "group-element last",
                "tooltip": "Block Applications Names (List Format)"
         }
       ]
   }
  

• apiOperations: You can fetch options for the select and multiselect fields from the API that you have defined in your operation. The parameters to this operation will be what users have entered in the configuration of this operation and the target field. apiOperations is not supported for the connector configuration.
  Note: To hide this operation in the from the Actions drop-down list in the Playbook Designer set "visible": false for this operation.
• supports_remote: Some connector actions cannot work when run on agents in a segmented network. Therefore, parameters of those actions must be marked as "supports_remote": False. For example:

  "title": "Enable abc service",
  "name": "abc_service",
  "supports_remote":  False
  

• conditional_output_schema: Support for multi-option or dynamic output schema in connectors. In every operation that requires the dynamic output schema, you must add conditional_output_schema as a key in an array of objects. Each object will contain the condition, which you should add within {{ }} and then you must add the corresponding output schema for each condition. It is recommended that you add a default schema, using "condition": "{{true}}", at the end of the defined conditions. The default schema will be used if none of the defined conditions are met.
  Notes:
  1. If you are using multiple conditions for evaluation such as a combination of && and ||, then you must wrap the whole condition within ().
     For example, {{(command === 'ls' || command === 'ssh' )}}
{{(command === 'ls' && port === 5895 )}}
  2. If you are using the condition with the checkbox field, add the condition as:
     {{ checkbox === true}}
  3. If you are using the condition with the multi-select field, add the condition as:
     {{multsel.toString() === (['option1','option2']).toString()}}
  4. If you are using the condition with the json field, add the condition as:
     {{jsonfieldname === '{\"key\":\"value\"}'}}
     For example, {{jsonfieldname === '{\"name\":\"fortinet\"}'}}
     Important: In order to support versions earlier to the 4.12.0 version, output_schema will also always be present.
     Example of a conditional_output_schema definition:
     

     {
            "conditional_output_schema": [                 
               {
                       "condition": "{{command === 'ls'}}",
                       "output_schema": {
                           "op_result": "",
                           "op_status": ""
                       }
                   },
                   {
                       "condition": "{{command === 'ssh'}}",
                       "output_schema": {
                           "result": "",
                           "status": ""
                       }
                   },                 
                   {
                       "condition": "{{true}}",
                       "output_schema": {
                           "result_default": "",
                           "status": ""
                       }
                   }
          ]
      }
     

connector.py

The connector.py class extends the base connector class and implements the check_health and execute functions. Following is a skeleton of this class:

from connectors.core.connector import Connector, get_logger
logger = get_logger('<connector_name>')

class Template(Connector):
   def execute(self, config, operation, params, **kwargs):
   	supported_operations = {'operation_1': function_template}
   	return supported_operations[operation](config, params)
   
   def check_health(self, input):
       return True

Notes:

• All imports for the connector files should be relative. For example, if your util.py is parallel to connector.py and you want to import util.py then you must import it as: import .util.
• get_logger is a utility function to initialize the logger. You can import get_logger from connectors.core.connector, and then declare the logger as logger = get_logger('<connector name>'). All the connector logs are written to the /var/log/cyops/cyops-integrations/connectors.log file and follow the format: %(asctime)s %(levelname)s %(name)s %(module)s %(funcName)s(): %(message)s.
• In addition to the execute and check_health functions, the following optional and additional functions are also available:
  on_add_config(self, config): Invoked when a new configuration is added to the connector. This is an optional function that can be overridden while setting up a new configuration.
  on_update_config(selfself, old_config, new_config): Invoked when a configuration is updated for the connector. This is an optional function that can be overridden while setting up a configuration edit function.
  on_delete_config(self, config): Invoked when a configuration is deleted from the connector. This is an optional function that can be overridden while setting up a configuration teardown function.
  on_activate(self, config) : Invoked when a configuration is activated.
  on_deactivate(self, config) : Invoked when a configuration is deactivated.
  on_teardown(self) : Invoked when a configuration is deleted. This is an optional function that can be overridden for dismantling a connector.
  You can use these functions to perform specific operations such as starting or stopping of a service at the relevant events. For example, you can use the on_add_config() function to start and stop a service when a configuration is added.

playbooks.json

The playbook.json file contains any playbook collection that you want to include with your connector. This could be a set of playbooks that demonstrate the usage of your connector.

images

The images directory must have the connector icon files. The 'icon_small_name' and 'icon_large_name' keys in the info.json must match the names of these icon files inside the images folder. Icon files can be in the .jpg or .png formats.

Once you have created all the files, bundle them into a .tgz file. For example, tar -czvf sampleConnector.tgz sampleConnector/.

requirements.txt and packages

If a connector requires additional python libraries, specify the libraries in the requirements.txt file.

If there is a dependency on any custom packages or if your instance does not have internet access to download packages from the internet, you can add the packages to the packages directory in the connector folder.

During a connector import, the framework first runs pip install -r requirements.txt followed by pip install <package> for every package in the packages directory. The commands are run in a separate thread and import is marked successful even when the dependency installation is still in progress. The dependency install logs are available at /var/log/cyops/cyops-integrations/pipinstall_<timestamp>.log on the FortiSOAR instance. If more than five dependency install log files get accumulated, the log files that are older than a day get deleted.

If a dependency install fails, then you can install them again by invoking the REST APIs directly. Contact FortiSOAR Support for more details on the APIs.

Importing a connector into FortiSOAR

Use the "Connector Store" to install and configure connectors in FortiSOAR 5.0.0 and later. To install a connector, you must be assigned a role that has a minimum of Create access to the Connectors module. To configure connectors into FortiSOAR, you must be assigned a role that has a minimum of Update access to the Connectors module. For more information about the Connector Store and the process of importing connectors into FortiSOAR, see the Introduction to connectors chapter.

Importing a connector into FortiSOAR prior to version 5.0.0

1. Log on to FortiSOAR.
2. On the left navigation pane, click Automation > Connectors and click Add Connector.
   
3. Drag-and-drop the connector package file or click Browse File to import the connector.tgz file.
   FortiSOAR will prompt you to enter the configuration inputs that you have defined in the info.json file.
   Note: You can install different versions of a connector, enabling you to reference a specific version of a connector from a playbook. If you want to replace all previous versions of the connector, ensure that you click the Delete all existing versions checkbox while importing the new version of the connector. If you do not click the Delete all existing versions checkbox, then a new version of the connector is added. You must ensure that your playbooks reference a correct and existing version of the connector.
   Following is a sample image:
   
   FortiSOAR displays the Uploading Connector message and then displays the Connector Configuration popup.
4. To configure the connector, the Connector Configuration popup enter the required configuration details.
   The configuration details and the details of the connector specified in the info.json file are stored in the FortiSOAR database.
   Note: You can add multiple configurations for your connector if you have more than one instance of your third-party server in your environment. You must, therefore, add a unique Name for each configuration.
   If you have previous versions of a connector and you are configuring a newer version of that connector, with the same configuration parameters, then FortiSOAR fetches the configuration and input parameters of the latest available version of that connector. For example, if you have 1.0.0, 2.0.0, and 2.3.0 versions of the Fortinet FortiSIEM connector and you are configuring the 2.3.0 version of the Fortinet FortiSIEM connector, then while configuring the 2.3.0 version, FortiSOAR will fetch the configuration and input parameters from the 2.0.0 version of the Fortinet FortiSIEM connector. You can review the configuration and input parameters, and then decide to change them or leave them unchanged.
   You can activate or deactivate a configured connector by clicking on the Activate Connector or Deactivate Connector Link.
   You can check the Mark As Default Configuration option to make the selected configuration, the default configuration of this connector, on the particular FortiSOAR instance.
   The password type fields in FortiSOAR include encryption and decryption. Passwords are encrypted before saving them into the database and decrypted when they are used in actions. In case of an upgrade, connectors that are already installed will work with stored passwords.
5. To save your configuration, click Save.
   To view the list of actions that can be performed by the connector, click the Actions tab.
   To view the playbook file that is bundled with the connector, click the Playbooks tab.
   You can optionally perform a Health Check by clicking the Refresh icon that is present in the Health Check bar. The Health Check checks if the configuration parameters you have specified are correct and if connectivity can be established to the specified server, endpoint or API.
   If all the details are correct and the connectivity to the server can be established, then on the Connectors page, Available is displayed in the health check dialog.
   If any or all the details are incorrect or if the connectivity to the server cannot be established then on the Connectors page, Disconnected is displayed in the health check dialog.
   You can also click the Refresh icon that is present in the Health Check bar to perform the health check at any time.

Reimporting a connector

After importing a connector, any changes done in the python files of the connector automatically get reflected the next time you run a command on the connector. However, if changes are made in the info.json, then you need to reimport the connector, using the following command, for the updates to take effect:
/opt/cyops-integrations/.env/bin/python /opt/cyops-integrations/integrations/manage.py reimport_connector -n <connector_name> -cv <connector_version> -migrate

If you only want to update the info.json changes and not retain the previous connector configuration, then you can omit the -migrate attribute as follows:
/opt/cyops-integrations/.env/bin/python /opt/cyops-integrations/integrations/manage.py reimport_connector -n <connector_name> -cv <connector_version>

You can also reimport all connectors at a single time using the following command by omitting the -n <connector_name> and -cv <connector_version> attributes as follows:
/opt/cyops-integrations/.env/bin/python /opt/cyops-integrations/integrations/manage.py reimport_connector -migrate

Check_Health function

The check_health function of the connector is invoked when you click the Refresh icon. This function takes the dictionary of the configuration parameters as the input. For example: {‘url’: ‘https://xyz.com’, 'user': 'admin', 'password': 'password'}.

You must throw the ConnectorError from the function if you want the check_health function to fail in a given scenario, such as issues with connectivity or with the provided credentials. To throw the ConnectorError, you must import it from the connectors.core.connector module as follows:

from connectors.core.connector import ConnectorError

Add connector operation to a playbook

Once you have completed configuring and deploying your connector, you can add a connector operation to a playbook, by adding the connector as a step in the playbook, as shown in the following image:

You can install different versions of a connector, and while adding a connector operation, you specify a specific version of a connector within a Playbook. In case you have installed multiple connectors, and if the version of the connector specified in the playbook is not found, then the playbook by default uses the latest version. FortiSOAR checks for the latest version of the connector in the format "major version.minor version.patch version". For example, version 2.0.2 is a later version than 1.2.0.

The execute function of connector.py is called when an operation on the connector is invoked. This function takes the dictionary containing the config, operation and params fields. For example:

{'config': {'password': password, 'server_url': 'https://xyz.com', 'user': 'admin'}, 
'params': {'input1': 'value1', 'input2': 'value2'}, 
'operation': 'function_template'}

The return from the execute function is set in the results.data variable of the playbook step. A sample execute function is present in the connector.py section.

The info.json contains output_schema, which defines the keys that are present in the output json on the execution of an operation. The info.json contains some common keys. However, the output json can have additional keys based on the input parameters. You can use these json keys to set the input for subsequent Playbook Steps, using Dynamic Values. For more information, see the Dynamic Values section in the "Playbooks Guide."

Configuring a connector to return a response

If you want to return a response from any action of a connector, then you can import the Result class from the connectors framework to your connector.py and then you can set message attributes such as status, message, etc.

Following is the python snippet for the same:

Updating a connector configuration using the update_connnector_config() function

The update_connnector_config() function can be internally called from any connector to update the configuration of any connector by providing the name and version of that connector. Steps to be followed for updating a connector configuration:

1. Import the update_connnector_config() function from /opt/cyops-integrations/integrations/connectors/core/utils.py.
2. Call the update_connnector_config function with the following parameters: <connector_name>, <connector_version>, <update_config>, and <config_id>.
   update_connnector_config(<connector_name>,<connector_version>,<update_config>,<config_id>)
   For example, update_connnector_config("imap","3.2.0",{"username":"csdamin","password":"csadminpwd"},"5785130913212321")
   Notes:
   If you do not provide the <config_id>, then the connector configuration that you have marked as default configuration will be updated.
   If you have not marked any connector configuration as the default configuration, and you have also not provided the <config_id>, then the following error is raised: No configuration found to update. Please add a config_id or mark a configuration as the default. in the /var/log/cyops/cyops-integrations/connectors.log. To resolve this error, either mark a connector configuration as the default configuration or provide the <config_id>.

Configuring the custom connector to support data ingestion

If your custom connector also supports data ingestion, you require to make the following updates so that your connector works with the ingestion wizard in the FortiSOAR:

1. Update the info.json file to contain the following:
   "ingestion_supported": true
   Also, specify the ingestion method(s) that is supported by your custom connector. FortiSOAR supports the following ingestion methods:
   "ingestion_modes": ["scheduled", "notification", "app_push"]
2. You must include sample ingestion playbooks with your custom connector. You can refer to the sample collection of ingestion-enabled connectors, such as FortiAnalyzer and FortiSIEM, which are included with FortiSOAR. Your sample ingestion playbooks must be created keeping the following points in mind:
   1. Each playbook that contributes to data ingestion must contain the following tags:
      <connector_name>, dataingestion
   2. Fetch Playbook: This playbook fetches sample data and is also used for the actual fetch operation once ingestion is configured. This playbook must be created considering the following:
      Contains an additional tag: fetch
      Contains a Set Variable step named "Configuration" that defines all user inputs needed for the playbook
      Returns results in "data" keys. {{vars.result.data}} of this playbook is what is presented to users as source data on the Field Mapping screen in the data ingestion wizard.
   3. Ingest Playbook: This is a wrapper playbook that is scheduled or called from the notification service. This playbook generally calls the fetch playbook as a reference and then loops over the result creating "alert" records in FortiSOAR.
      The step that creates records should be named "Create Record". The ingestion wizard reads this step and displays the default mappings from the source data to the record created in FortiSOAR. If the step uses a 'bulk' Create Record step, the mappings are shown using the item variable. This playbook must have the following additional tags: fetch, create.
   4. Create Record Playbook: If record creation is a multi-step process and cannot use the single "Create Record" step in bulk, then you can move record creation to a separate playbook that is invoked in a loop after fetch. If this is the case, then the "Create Record" playbook must have the create tag and this tag must not be present in the "Ingest" playbook. This playbook must be created considering the following:
      Defines an input parameter "sourcedata"
      Contain only one Create Record and the step name must also be named "Create Record"
      "Create Record" step works on the "sourcedata" input
      Note: Multiple tags can be applied on the same playbook. However, you need to ensure that the "fetch" playbook does not also create records, since otherwise it will also be called for fetching sample data.

FortiSOAR Connector SDK

The FortiSOAR Connector SDK consists of a simulator for writing and testing your connector. You can use this CLI to help you develop your custom connector, without having a ForiSOAR™ instance.

For information on how to setup and use the FortiSOAR Connector SDK, see Fortinet Developer Network (FNDN). You can also download the Connector SDK from FNDN and use it to develop a custom connector. You must log into FNDN to access the Connector SDK.