Block access to LLM applications using keywords and FQDN
Large language models (LLMs), such as GPT, are a type of Generative AI (GenAI) and are widely used in applications like chatbots. This configuration will block HTTPS upload traffic to LLM applications that include sensitive keywords. The pre-defined data type, keyword, is used in the Data Loss Prevention (DLP) dictionary.
To entirely prevent access to LLM applications, use a web filter profile with the FortiGuard Artificial Intelligence Technology category set to block. See Configuring web filter profiles to block AI and cryptocurrency for more information. To confirm that the URL of the LLM application that you need to block is in the Artificial Intelligence Technology category. The URL category can be verified using the FortiGuard Web Filter Lookup. |
When utilizing commonly-used SSL-encrypted protocols such as HTTPS, SMTPS, POP3S, IMAPS, and FTPS, SSL inspection must be set to Deep Inspection. See Deep inspection for more information. Additionally, the client machine must have the corresponding deep inspection Certificate Authority (CA) certificate installed. |
Example
In this example, a user is conducting a search on an LLM application in the Chrome browser on an office computer, using a sensitive keyword that has been configured in the DLP dictionary. The FortiGate intercepts this traffic using deep inspection and prevents the search that contains sensitive keywords because it matches the DLP profile that has been set up on this FortiGate.
When a sensitive keyword is included in HTTPS upload traffic, the request is blocked and a DLP log is generated.
To block HTTPS upload traffic that includes sensitive keywords in the GUI:
-
Configure the DLP dictionary:
-
Go to Security Profiles > Data Loss Prevention, select the Dictionaries tab, then click Create New.
-
Set Name to llmapps.
-
In the Dictionary Entries table click Create New:
-
Set Type to keyword.
-
Set Pattern to fortinet.
-
Enable Case sensitive.
-
Click OK.
-
-
Repeat step c and set Pattern to source code.
-
Click OK.
-
-
Configure the DLP sensor:
-
Go to Security Profiles > Data Loss Prevention, select the Sensors tab, then click Create New.
-
Set Name to llmapps.
-
In the Sensor Entries section, click Create New:
-
Set the Dictionary to llmapps then click OK.
-
-
Click OK.
-
-
Configure the DLP profile:
-
Go to Security Profiles > Data Loss Prevention, select the Profiles tab, then click Create New.
-
Set Name to llmapps.
-
In the Rules section, click Create New:
-
Configure the following settings:
Name
llmapps1
Sensors
llmapps
Severity
Critical
Action
Block
Type
File
File type
builtin-patterns
Protocol
HTTP-POST
-
Click OK.
-
-
In the Rules section, click Create New again:
-
Configure the following settings:
Name
llmapps2
Sensors
llmapps
Severity
Critical
Action
Block
Type
Message
Protocol
HTTP-POST
-
Click OK.
-
-
Click OK to save the profile.
-
Unset the file type option to enable filtering of all file types, including unknown ones:
config dlp profile edit "llmapps" config rule edit 1 unset file-type next end next end
-
-
Configure the firewall address for the LLM application:
-
Go to Policy & Objects > Addresses, select the Standard tab, then click Create New.
-
Set the following:
Name
<name>
Type
FQDN
FQDN
See FQDN for the FQDN on the specific LLM application.
-
Click OK.
-
-
Add the firewall address to a group:
-
Go to Policy & Objects > Addresses, select the Address Group tab, then click Create New.
-
Set the following:
Group name
llmapps
Members
ChatGpt, AmazonQ, Gemini
-
-
Add the DLP profile and the address group to a firewall policy:
-
Go to Policy & Objects > Firewall Policy and click Create New.
-
Set the Inspection Mode to Proxy-based.
-
Set the Destination to llmapps.
-
In the Security Profiles section:
-
Enable Application control and select the profile that QUIC is blocked in. In this example, the default profile is selected.
-
Enable DLP Profile and select llmapps.
-
-
Set SSL Inspection to deep-inspection.
-
Configure the other settings as needed.
-
Click OK.
-
To block HTTPS upload traffic that includes sensitive keywords in the CLI:
-
Configure the DLP dictionary:
config dlp dictionary edit "llmapps" config entries edit 1 set type "keyword" set pattern "source code" set ignore-case enable next edit 2 set type "keyword" set pattern "fortinet" set ignore-case enable next end next end
-
Configure the DLP sensor:
config dlp sensor edit "amazonQ" config entries edit 1 set dictionary "llmapps" next end next end
-
Configure the DLP profile:
config dlp profile edit "llmapps" set feature-set proxy config rule edit 1 set name "llmapps1" set severity critical set proto http-post set filter-by sensor set sensor "llmapps" set action block next edit 2 set name "llmapps2" set type message set proto http-post set filter-by sensor set sensor "llmapps" set action block next end next end
-
Configure the firewall address for the LLM application:
config firewall address edit <name> set type fqdn set fqdn <string> next end
See FQDN for the FQDN on the specific LLM application.
-
Add the firewall addresses to a group:
config firewall addrgrp edit "llmapps" set member "ChatGpt" "AmazonQ" "Gemini" next end
-
Add the DLP profile and the FQDN address to a firewall policy:
config firewall policy edit 1 set name "llmapps " set srcintf "port2" set dstintf "port1" set action accept set srcaddr "all" set dstaddr "llmapps" set schedule "always" set service "ALL" set utm-status enable set inspection-mode proxy set ssl-ssh-profile "deep-inspection" set dlp-profile "llmapps" set application-list "default" set nat enable next end
FQDN
A firewall address should be created for each LLM application:
LLM application |
FQDN |
---|---|
ChatGPT |
chat.openai.com |
Amazon Q |
qbusiness.us-west-2.api.aws The FQDN for the Amazon Q application can vary depending on the region. In this example, the application is deployed in the US West (Oregon) region. See Service regions and endpoints. |
Gemini |
gemini.google.com |
Verification
ChatGPT:
-
Visit https://chat.openai.com.
-
Search for any phrase that includes keywords set up in the DLP dictionary.
-
Verify that the request failed and an error was generated.
-
Review the log that was generated when the attempt was made to send an HTTP POST request containing sensitive keywords:
1: date=2024-03-15 time=09:59:35 eventtime=1710453575538415503 tz="+1200" logid="0954024576" type="utm" subtype="dlp" eventtype="dlp" level="warning" vd="root" ruleid=1 rulename="llmapps1" dlpextra="Sensor 'llmapps' matching any: ('llmapps'=1) >= 1; match." filtertype="sensor" filtercat="file" severity="critical" policyid=1 poluuid="eea32b46-db4e-51ee-92a9-b46e5580db33" policytype="policy" sessionid=69254 epoch=424445846 eventid=1 srcip=13.13.13.13 srcport=56747 srccountry="United States" srcintf="port2" srcintfrole="undefined" srcuuid="d2f06fda-15e7-51ee-0d22-faaf5170dad2" dstip=104.18.37.228 dstport=443 dstcountry="United States" dstintf="port1" dstintfrole="undefined"
Amazon Q:
-
Visit your deployed Amazon Q application at the deployed URL link.
-
Search for any phrase that includes keywords set up in the DLP dictionary.
-
Verify that the request failed and an error was generated.
-
Review the log that was generated when the attempt was made to send an HTTP POST request containing sensitive keywords:
1: date=2024-03-15 time=09:50:29 eventtime=1710453029263609219 tz="+1200" logid="0954024576" type="utm" subtype="dlp" eventtype="dlp" level="warning" vd="root" ruleid=1 rulename="llmapps1" dlpextra="Sensor 'llmapps' matching any: ('llmapps'=1) >= 1; match." filtertype="sensor" filtercat="file" severity="critical" policyid=1 poluuid="eea32b46-db4e-51ee-92a9-b46e5580db33" policytype="policy" sessionid=68636 epoch=424445675 eventid=1 srcip=13.13.13.13 srcport=56589 srccountry="United States" srcintf="port2" srcintfrole="undefined" srcuuid="d2f06fda-15e7-51ee-0d22-faaf5170dad2" dstip=44.232.12.199 dstport=443 dstcountry="United States" dstintf="port1" dstintfrole="undefined" dstuuid="e7274712-db4e-51ee-bbeb-d74c8cba6747" proto=6 service="HTTPS" filetype="unknown" direction="outgoing" action="block" hostname="qbusiness.us-west-2.api.aws" url="https://qbusiness.us-west-2.api.aws/applications/a2599147-8fb4-4a76-9b65-cf5b4730aa9f/conversations?sync&userId=ssukhdeep@fortinet.com" agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:123.0) Gecko/20100101 Firefox/123.0" httpmethod="POST" referralurl="https://bgiotxzy.chat.qbusiness.us-west-2.on.aws/" filename="conversations" filesize=104 profile="llmapps"
Gemini:
-
Visit https://gemini.google.com .
-
Search for any phrase that includes keywords set up in the DLP dictionary.
-
Verify that the request failed and an error was generated.
-
Review the log that was generated when the attempt was made to send an HTTP POST request containing sensitive keywords:
1: date=2024-03-15 time=12:46:08 eventtime=1710463568053453203 tz="+1200" logid="0954024576" type="utm" subtype="dlp" eventtype="dlp" level="warning" vd="root" ruleid=1 rulename="llmapps1" dlpextra="Sensor 'llmapps' matching any: ('llmapps'=1) >= 1; match." filtertype="sensor" filtercat="file" severity="critical" policyid=1 poluuid="eea32b46-db4e-51ee-92a9-b46e5580db33" policytype="policy" sessionid=77832 epoch=424449372 eventid=1 srcip=13.13.13.13 srcport=58137 srccountry="United States" srcintf="port2" srcintfrole="undefined" srcuuid="d2f06fda-15e7-51ee-0d22-faaf5170dad2" dstip=142.251.33.110 dstport=443 dstcountry="United States" dstintf="port1" dstintfrole="undefined" dstuuid="d2f06fda-15e7-51ee-0d22-faaf5170dad2" proto=6 service="HTTPS" filetype="unknown" direction="outgoing" action="block" hostname="gemini.google.com" url="https://gemini.google.com/_/BardChatUi/data/assistant.lamda.BardFrontendService/StreamGenerate?bl=boq_assistant-bard-web-server_20240313.09_p0&f.sid=2103257702826212605&hl=en&_reqid=1474614&rt=c" agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" httpmethod="POST" referralurl="https://gemini.google.com/" filename="StreamGenerate" filesize=2211 profile="llmapps"