Fortinet white logo
Fortinet white logo

OCI Administration Guide

Troubleshooting

Troubleshooting

To validate your HA configuration sync you can issue:

diagnose sys ha checksum show

OCI components in FortiOS come with their own daemon, including debug output. This can be invoked with:

diagnose debug application ocid -99

You can display diagnose commands with:

diagnose test application ocid -1

1. show HA stats

2. SDN api test

3. HA api test

4. filter list test

99. restart

You can verify that the following diagnose command works for the ocid daemon:

On FortiGate A:

diag test application ocid 1

ocid stats:

master: 1

On FortiGate B:

diag test application ocid 1

ocid stats:

master: 0

SDN api test is practical to see whether your sdn-connector configuration can successfully authenticate and issue commands to OCI Management.

Running HA api test on production environments is not recommended. This may lead your cluster to a mixed state. Use it only to see whether ocid daemon successfully sends failover commands to OCI Management.

If you have performed any modifications to your CLI configuration, restart your ocid daemon by running the following commands:

diag test application ocid 99

ocid start

By default, all configuration between firewalls is synchronized. Since some settings, especially NAT, are node-specific, you may want to disable synchronization.

config system ha

set sync-config disable

end

During a successful HA failover event, the secondary FortiGate-VM takes over the private IP address from the active unit to the passive unit. The following shows the sample debug output in this scenario:

FGVM8VTM19000449 # diag debug enable

FGVM8VTM19000449 # diag debug application ocid -1

Debug messages will be on for 30 minutes.

FGVM8VTM19000449 # HA event

Become HA master

ocid collect vnics info for instance fgtvminstance-2

vnic id(1/4): ocid1.vnic.oc1.ca-toronto-1.ab2g6ljraiheu5bqvg5riy4rsngg2lm6z766glghhlneqjld3gcpquuhlv5a

vnic id(2/4): ocid1.vnic.oc1.ca-toronto-1.ab2g6ljrhxf63fvlacjnyl6del3vzo42g5cjyvlczvosxuc5dtn4zqrnwdsa

vnic id(3/4): ocid1.vnic.oc1.ca-toronto-1.ab2g6ljr3hmbq675vbgjbuwn2aywjhonqmwb5slxjitwy4pyw3fipa2wzwpq

vnic id(4/4): ocid1.vnic.oc1.ca-toronto-1.ab2g6ljruyxpzi4db2tjet45gix3qauwwgnvf3pbsjcvbd337rgr7ygyy4ka

ocid fail over private ip: 10.0.12.3

private ip 10.0.12.3 is attached in remote instance

attaching private ip 10.0.12.3 to local vnic (ocid1.vnic.oc1.ca-toronto-1.ab2g6ljraiheu5bqvg5riy4rsngg2lm6z766glghhlneqjld3gcpquuhlv5a)

updating private ip with data: {"vnicId": "ocid1.vnic.oc1.ca-toronto-1.ab2g6ljraiheu5bqvg5riy4rsngg2lm6z766glghhlneqjld3gcpquuhlv5a"}

moving private ip 10.0.12.3 to local successfully

ocid fail over private ip: 10.0.12.5

private ip 10.0.12.5 is attached in remote instance

attaching private ip 10.0.12.5 to local vnic (ocid1.vnic.oc1.ca-toronto-1.ab2g6ljraiheu5bqvg5riy4rsngg2lm6z766glghhlneqjld3gcpquuhlv5a)

updating private ip with data: {"vnicId": "ocid1.vnic.oc1.ca-toronto-1.ab2g6ljraiheu5bqvg5riy4rsngg2lm6z766glghhlneqjld3gcpquuhlv5a"}

moving private ip 10.0.12.5 to local successfully

ocid fail over private ip: 10.0.8.10

private ip 10.0.8.10 is attached in remote instance

attaching private ip 10.0.8.10 to local vnic (ocid1.vnic.oc1.ca-toronto-1.ab2g6ljr3hmbq675vbgjbuwn2aywjhonqmwb5slxjitwy4pyw3fipa2wzwpq)

updating private ip with data: {"vnicId": "ocid1.vnic.oc1.ca-toronto-1.ab2g6ljr3hmbq675vbgjbuwn2aywjhonqmwb5slxjitwy4pyw3fipa2wzwpq"}

moving private ip 10.0.8.10 to local successfully

To access FortiOS via the console:

If the instance is malfunctioning, you can attempt access to the instance via the console for troubleshooting.

  1. Create the console connection for an instance:
    1. In the OCI console, go to Core Infrastructure > Compute > Instances. Select the desired instance name.
    2. Go to Resources > Console Connections. Click Create Console Connection.
    3. Specify the public key (.pub) portion for the SSH key. You can browse to a public key file on your computer or paste your public key into the text field. Then, click Create Console Connection. When the console connection has been created and is available, the status changes to ACTIVE.
  2. Connect to FortiOS via the console using OpenSSH on macOS or Linux:
    1. Click the Actions icon, then click Connect with SSH.
    2. In the Connect with SSH dialog, click Copy to copy the string to your clipboard.

    3. Use the string to connect to the FortiGate-VM instance. Ensure that you specify the correct SSH key and use -i:

      ssh -i id_rsa -o ProxyCommand='ssh -i id_rsa -W %h:%p -p 443 …..

Troubleshooting

Troubleshooting

To validate your HA configuration sync you can issue:

diagnose sys ha checksum show

OCI components in FortiOS come with their own daemon, including debug output. This can be invoked with:

diagnose debug application ocid -99

You can display diagnose commands with:

diagnose test application ocid -1

1. show HA stats

2. SDN api test

3. HA api test

4. filter list test

99. restart

You can verify that the following diagnose command works for the ocid daemon:

On FortiGate A:

diag test application ocid 1

ocid stats:

master: 1

On FortiGate B:

diag test application ocid 1

ocid stats:

master: 0

SDN api test is practical to see whether your sdn-connector configuration can successfully authenticate and issue commands to OCI Management.

Running HA api test on production environments is not recommended. This may lead your cluster to a mixed state. Use it only to see whether ocid daemon successfully sends failover commands to OCI Management.

If you have performed any modifications to your CLI configuration, restart your ocid daemon by running the following commands:

diag test application ocid 99

ocid start

By default, all configuration between firewalls is synchronized. Since some settings, especially NAT, are node-specific, you may want to disable synchronization.

config system ha

set sync-config disable

end

During a successful HA failover event, the secondary FortiGate-VM takes over the private IP address from the active unit to the passive unit. The following shows the sample debug output in this scenario:

FGVM8VTM19000449 # diag debug enable

FGVM8VTM19000449 # diag debug application ocid -1

Debug messages will be on for 30 minutes.

FGVM8VTM19000449 # HA event

Become HA master

ocid collect vnics info for instance fgtvminstance-2

vnic id(1/4): ocid1.vnic.oc1.ca-toronto-1.ab2g6ljraiheu5bqvg5riy4rsngg2lm6z766glghhlneqjld3gcpquuhlv5a

vnic id(2/4): ocid1.vnic.oc1.ca-toronto-1.ab2g6ljrhxf63fvlacjnyl6del3vzo42g5cjyvlczvosxuc5dtn4zqrnwdsa

vnic id(3/4): ocid1.vnic.oc1.ca-toronto-1.ab2g6ljr3hmbq675vbgjbuwn2aywjhonqmwb5slxjitwy4pyw3fipa2wzwpq

vnic id(4/4): ocid1.vnic.oc1.ca-toronto-1.ab2g6ljruyxpzi4db2tjet45gix3qauwwgnvf3pbsjcvbd337rgr7ygyy4ka

ocid fail over private ip: 10.0.12.3

private ip 10.0.12.3 is attached in remote instance

attaching private ip 10.0.12.3 to local vnic (ocid1.vnic.oc1.ca-toronto-1.ab2g6ljraiheu5bqvg5riy4rsngg2lm6z766glghhlneqjld3gcpquuhlv5a)

updating private ip with data: {"vnicId": "ocid1.vnic.oc1.ca-toronto-1.ab2g6ljraiheu5bqvg5riy4rsngg2lm6z766glghhlneqjld3gcpquuhlv5a"}

moving private ip 10.0.12.3 to local successfully

ocid fail over private ip: 10.0.12.5

private ip 10.0.12.5 is attached in remote instance

attaching private ip 10.0.12.5 to local vnic (ocid1.vnic.oc1.ca-toronto-1.ab2g6ljraiheu5bqvg5riy4rsngg2lm6z766glghhlneqjld3gcpquuhlv5a)

updating private ip with data: {"vnicId": "ocid1.vnic.oc1.ca-toronto-1.ab2g6ljraiheu5bqvg5riy4rsngg2lm6z766glghhlneqjld3gcpquuhlv5a"}

moving private ip 10.0.12.5 to local successfully

ocid fail over private ip: 10.0.8.10

private ip 10.0.8.10 is attached in remote instance

attaching private ip 10.0.8.10 to local vnic (ocid1.vnic.oc1.ca-toronto-1.ab2g6ljr3hmbq675vbgjbuwn2aywjhonqmwb5slxjitwy4pyw3fipa2wzwpq)

updating private ip with data: {"vnicId": "ocid1.vnic.oc1.ca-toronto-1.ab2g6ljr3hmbq675vbgjbuwn2aywjhonqmwb5slxjitwy4pyw3fipa2wzwpq"}

moving private ip 10.0.8.10 to local successfully

To access FortiOS via the console:

If the instance is malfunctioning, you can attempt access to the instance via the console for troubleshooting.

  1. Create the console connection for an instance:
    1. In the OCI console, go to Core Infrastructure > Compute > Instances. Select the desired instance name.
    2. Go to Resources > Console Connections. Click Create Console Connection.
    3. Specify the public key (.pub) portion for the SSH key. You can browse to a public key file on your computer or paste your public key into the text field. Then, click Create Console Connection. When the console connection has been created and is available, the status changes to ACTIVE.
  2. Connect to FortiOS via the console using OpenSSH on macOS or Linux:
    1. Click the Actions icon, then click Connect with SSH.
    2. In the Connect with SSH dialog, click Copy to copy the string to your clipboard.

    3. Use the string to connect to the FortiGate-VM instance. Ensure that you specify the correct SSH key and use -i:

      ssh -i id_rsa -o ProxyCommand='ssh -i id_rsa -W %h:%p -p 443 …..