Allow grace period for FortiFlex to begin passing traffic upon activation

This enhancement allows for a two-hour grace period for FortiFlex to begin passing traffic upon retrieving the license from FortiCare without VM entitlement verification from FortiGuard Distribution Servers (FDS). In the past, after retrieving the license from FortiCare, traffic was not allowed to pass until the license entitlement was verified with FDS. Since FDS must communicate with FortiCare to retrieve license and entitlement updates, this delayed the entitlement check from the FortiGate. Such delays negatively impacted autoscaling and on-demand instances.

The following shows the topology for this enhancement. The topology shows two step 2s because they happen concurrently.

The topology illustrates the following process:

1. The user generates the FortiFlex license.
2. FDS pulls the registration data from FortiCare, updates the FortiGuard Developer Network (FDN) database, and pushes updates to other systems. The user immediately registers the FortiFlex token on the FortiGate.
3. The FortiGate reaches FortiCare to obtain the license. The grace period immediately starts, and the FortiFlex can pass traffic.
4. The FortiGate synchronizes entitlement from FDS. The FortiGate license becomes valid.

The following scenarios illustrate how the grace period works in production.

Scenario 1

In this scenario, the FortiGate-VM can reach FDS and FortiCare. The FortiGate-VM activates a newly generated FortiFlex license token before FDS synchronizes the license information from FortiCare. The FortiGate-VM is granted the two-hour license grace period.

1. Create a FortiGate-VM with an evaluation license. The following shows the get system status output at this point:

   Version: FortiGate-VM64 v7.2.0,build1115,220218 (interim)

   Serial-Number: FGVMEVVEWEABZJ10

2. Generate a new FortiFlex license on FortiCare and activate the FortiFlex license token in FortiOS immediately:

   FGT-TEMP68 # exec vm-license AC6A134D807CDA4B75F8

   This operation will reboot the system !

   Do you want to continue? (y/n)y

3. The FortiGate-VM sets the VM license status as Grace Period before it can validate the license with FDS. The following shows the get system status output at this point:

   Version: FortiGate-VM64 v7.2.0,build1115,220218 (interim)
   ...
   Serial-Number: FGVMMLTM22000386 
   License Status: Grace Period 
   License Expiration Date: 2022-08-03
   VM Resources: 1 CPU/4 allowed, 2007 MB RAM
   Log hard disk: Available

   The following shows the diagnose debug vm-print-license output at this point:

   SerialNumber: FGVMMLTM22000386
   CreateDate: Fri Feb 18 16:30:02 2022
   License expires: Wed Aug 3 00:00:00 2022
   Default Contract: FMWR:6:20220218:20220803,ENHN:20:20220218:20220803,COMP:20:20220218:20220803,AVDB:6:20220218:20220803,NIDS:6:20220218:20220803,FURL:6:20220218:20220803,SPAM:6:20220218:20220803,VMLS:6:20220218:20220803:4
   Key: yes
   Cert: yes
   Key2: yes
   Cert2: yes
   Model: ML (21)
   CPU: 4 (subscription:4)
   MEM: 2147483647
   VDOM license:
     permanent: 1
     subscription: 0
   Grace period: 119 min 34 sec

   The following shows the diagnose hardware sysinfo vm full output at this point:

   UUID: 4213ad10566d26bf8128bcce13ee457c
   valid: 1
   status: 6
   code: 400 
   warn: 0
   copy: 0
   received: 4294939603
   warning: 4294939603
   recv: 202202181631
   dup:

4. FDS synchronizes the license information from FortiCare. The followwing shows the output at this point:

   Version: FortiGate-VM64 v7.2.0,build1115,220218 (interim)
   ...
   Serial-Number: FGVMMLTM22000386
   License Status: Valid
   License Expiration Date: 2022-08-03
   VM Resources: 1 CPU/4 allowed, 2007 MB RAM
   Max number of virtual domains: 1
   Virtual domains status: 1 in NAT mode, 0 in TP mode
   Virtual domain configuration: disable

   The following shows the diagnose debug vm-print-license output at this point:

   SerialNumber: FGVMMLTM22000386
   CreateDate: Fri Feb 18 16:30:02 2022
   License expires: Wed Aug 3 00:00:00 2022
   Default Contract: FMWR:6:20220218:20220803,ENHN:20:20220218:20220803,COMP:20:20220218:20220803,AVDB:6:20220218:20220803,NIDS:6:20220218:20220803,FURL:6:20220218:20220803,SPAM:6:20220218:20220803,VMLS:6:20220218:20220803:4
   Key: yes
   Cert: yes
   Key2: yes
   Cert2: yes
   Model: ML (21)
   CPU: 4 (subscription:4)
   MEM: 2147483647
   VDOM license:
     permanent: 1
     subscription: 0

   The following shows the diagnose hardware sysinfo vm full output at this point:

   UUID: 4213ad10566d26bf8128bcce13ee457c
   valid: 1
   status: 1
   code: 200
   warn: 0
   copy: 0
   received: 4294939603
   warning: 4294939603
   recv: 202202181631
   dup:

5. You can check the license logs using the following commands:

   execute log filter category event
   execute log filter field service license
   execute log display
   3 logs found.
   3 logs returned.
   
   1: date=2022-02-18 time=08:31:30 eventtime=1645201890437418122 tz="-0800" logid="0100022804" type="event" subtype="system" level="critical" vd="root" logdesc="License status changed" service="license" sn="FGVMMLTM22000386" status="VALID" msg="License status changed to VALID"
   
   2: date=2022-02-18 time=08:30:39 eventtime=1645201838290517060 tz="-0800" logid="0100022804" type="event" subtype="system" level="critical" vd="root" logdesc="License status changed" service="license" sn="FGVMMLTM22000386" status="VALID" msg="License is in grace period"
   
   3: date=2022-02-18 time=08:26:27 eventtime=1645201586895849932 tz="-0800" logid="0100022804" type="event" subtype="system" level="critical" vd="root" logdesc="License status changed" service="license" sn="FGVMEVVEWEABZJ10" status="VALID" msg="License status changed to VALID"

Scenario 2

In this scenario, the FortiGate-VM cannot reach FDS but can reach FortiCare. When the FortiGate-VM receives the license file from FortiCare with the token, the two-hour grace period begins.

This scenario is unlikely to happen in production. If a FortiGate-VM can reach FortiCare, it can also likely reach FDS. This scenario illustrates what occurs if the two-hour grace period passes without communication with FDS.

The following shows the license logs in this scenario during the two-hour grace period:

2: date=2022-02-17 time=22:57:43 eventtime=1645167462076672946 tz="-0800" logid="0100022804" type="event" subtype="system" level="critical" vd="root" logdesc="License status changed" service="license" sn="FGVMMLTM123123" status="VALID" msg="License is in grace period"

The following shows the license logs in this scenario after the two-hour grace period has passed and the FortiGate-VM still cannot reach FDN, and the license status changes to invalid:

1: date=2022-02-18 time=00:57:45 eventtime=1645174666108212880 tz="-0800" logid="0100022804" type="event" subtype="system" level="critical" vd="root" logdesc="License status changed" service="license" sn="FGVMMLTM22000351" status="INVALID" msg="License status changed to INVALID"

The following shows the output from diagnose sys vd list | grep index after the two-hour grace period has passed and the FortiGate-VM still cannot reach FDN, and the license status changes to invalid:

name=root/root index=0 disabled fib_ver=22 rpdb_ver=0 use=144 rt_num=26 asym_rt=0 sip_helper=0, sip_nat_trace=1, mc_fwd=0, mc_ttl_nc=0, tpmc_sk_pl=0

name=vsys_ha/vsys_ha index=1 enabled fib_ver=5 rpdb_ver=1 use=75 rt_num=0 asym_rt=0 sip_helper=0, sip_nat_trace=1, mc_fwd=0, mc_ttl_nc=0, tpmc_sk_pl=0

name=vsys_fgfm/vsys_fgfm index=2 enabled fib_ver=4 rpdb_ver=0 use=72 rt_num=0 asym_rt=0 sip_helper=0, sip_nat_trace=1, mc_fwd=0, mc_ttl_nc=0, tpmc_sk_pl=0