Fortinet black logo

High Availability and Disaster Recovery Procedures - EventDB

Troubleshooting Disaster Recovery

Troubleshooting Disaster Recovery

GUI

While Replication is being set up, you can see the progress in the GUI in two locations.

  1. ADMIN > License > Nodes > Add DR
    or

  2. Jobs/Errors > Jobs > filter on 6.3.1-DR

Backend Logs

On both the Primary and Secondary nodes, use the cat command to view the backend logs (/opt/phoenix/log/phoenix.log):

cat /opt/phoenix/config/phoenix.log

Successful Enablement of Disaster Recovery of the Primary node

2021-08-20T16:26:22.186396-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7398,[phLogDetail]=631-DR, Step 1: check command type

2021-08-20T16:26:22.186451-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7407,[phLogDetail]=631-DR, Step 2: check command data

2021-08-20T16:26:22.186475-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7414,[phLogDetail]=631-DR, Step 3: load replication setting

2021-08-20T16:26:22.222539-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7444,[phLogDetail]=631-DR-init-primary, Step 1: Saving Disaster Recovery Settings on Primary

2021-08-20T16:26:22.252896-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7457,[phLogDetail]=631-DR-init-primary, Step 2: Saving peer SSH public key on Primary

2021-08-20T16:26:22.291143-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7471,[phLogDetail]=631-DR-init-primary, Step 3: Updating SSH Configuration on Primary

2021-08-20T16:26:22.328987-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7486,[phLogDetail]=631-DR-init-primary, Step 4: Updating SSH Known Hosts on Primary

2021-08-20T16:26:22.503675-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7508,[phLogDetail]=631-DR-init-primary, Step 5: Initializing Disaster Recovery

2021-08-20T16:26:22.577513-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7567,[phLogDetail]=631-DR-init-primary, Step 5.1: Running phinitprimary script on Primary

2021-08-20T16:26:23.521939-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7582,[phLogDetail]=631-DR-init-primary, Step 5.2: Sending Task to Secondary

2021-08-20T16:26:23.579504-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7619,[phLogDetail]=631-DR-init-primary, Step 5.3: Initializing Secondary

2021-08-20T16:26:23.620795-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 1

2021-08-20T16:26:23.644367-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 20

2021-08-20T16:26:23.671393-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 30

2021-08-20T16:26:23.709927-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 40

2021-08-20T16:26:23.740996-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 50

2021-08-20T16:29:01.385371-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 80

2021-08-20T16:29:01.454035-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7634,[phLogDetail]=631-DR-init-primary, Step 5.5: Secondary Setup complete

2021-08-20T16:29:01.454250-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7515,[phLogDetail]=631-DR-init-primary, Step 6: Waiting for App Server to come back up on Primary

2021-08-20T16:29:01.509276-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7523,[phLogDetail]=631-DR-init-primary, Step 7: Waiting for Secondary App Server to update Service Passwords

2021-08-20T16:29:01.797224-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=8187,[phLogDetail]=631-DR-init-primary, Step 7.1: get sevice user

2021-08-20T16:29:01.797257-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=8195,[phLogDetail]=631-DR-init-primary, Step 7.2: get secondary host

2021-08-20T16:29:01.797440-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=8214,[phLogDetail]=631-DR-init-primary, Step 7.3: send task updating Svc password to secondary

2021-08-20T16:29:01.882229-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=8224,[phLogDetail]=631-DR-init-primary, Step 7.4: finish updating Svc password on secondary

2021-08-20T16:29:01.884157-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7541,[phLogDetail]=631-DR-init-primary, Step 8: Restarting processes on Primary

2021-08-20T16:29:01.923315-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7549,[phLogDetail]=631-DR-init-primary, Step 9: Broadcasting DR tasks to all processes on Primary

2021-08-20T16:29:01.962740-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7559,[phLogDetail]=631-DR-init-primary, Step 10: Disaster Recovery setup complete

2021-08-20T16:29:02.169745-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7432,[phLogDetail]=631-DR, Step 4: initializing DR finished.

2021-08-20T17:03:54.103262-07:00 va3005 phMonitorSupervisor[7384]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7712,[phLogDetail]=631-DR-init-secondary, Step 1: Initalize secondary From Primary server

2021-08-20T17:03:54.103449-07:00 va3005 phMonitorSupervisor[7384]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7733,[phLogDetail]=631-DR-init-secondary, Step 2: parse task info

2021-08-20T17:03:54.103689-07:00 va3005 phMonitorSupervisor[7384]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7757,[phLogDetail]=631-DR-init-secondary, Step 3: update SSH keys of primary on secondary

2021-08-20T17:03:54.115290-07:00 va3005 phMonitorSupervisor[7384]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7775,[phLogDetail]=631-DR-init-secondary, Step 4: update SSH configurations on secondary

2021-08-20T17:03:54.126466-07:00 va3005 phMonitorSupervisor[7384]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7793,[phLogDetail]=631-DR-init-secondary, Step 5: update SSH known hosts for primary on secondary

2021-08-20T17:03:54.148876-07:00 va3005 phMonitorSupervisor[7384]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7817,[phLogDetail]=631-DR-init-secondary, Step 6: run phinitsecondary script

2021-08-20T17:06:29.865977-07:00 va3005 phMonitorSupervisor[7384]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7881,[phLogDetail]=631-DR-init-secondary, Step 10: finish initialize secondary

Successful Enablement of Disaster Recovery of the Secondary node

2021-08-20T16:29:29.146469-07:00 andy-super-ipv6 phMonitorSupervisor[11936]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7712,[phLogDetail]=631-DR-init-secondary, Step 1: Initalize secondary From Primary server

2021-08-20T16:29:29.146829-07:00 andy-super-ipv6 phMonitorSupervisor[11936]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7733,[phLogDetail]=631-DR-init-secondary, Step 2: parse task info

2021-08-20T16:29:29.147537-07:00 andy-super-ipv6 phMonitorSupervisor[11936]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7757,[phLogDetail]=631-DR-init-secondary, Step 3: update SSH keys of primary on secondary

2021-08-20T16:29:29.160266-07:00 andy-super-ipv6 phMonitorSupervisor[11936]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7775,[phLogDetail]=631-DR-init-secondary, Step 4: update SSH configurations on secondary

2021-08-20T16:29:29.172964-07:00 andy-super-ipv6 phMonitorSupervisor[11936]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7793,[phLogDetail]=631-DR-init-secondary, Step 5: update SSH known hosts for primary on secondary

2021-08-20T16:29:29.287039-07:00 andy-super-ipv6 phMonitorSupervisor[11936]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7817,[phLogDetail]=631-DR-init-secondary, Step 6: run phinitsecondary script

2021-08-20T16:32:06.952364-07:00 andy-super-ipv6 phMonitorSupervisor[11936]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7881,[phLogDetail]=631-DR-init-secondary, Step 10: finish initialize secondary

2021-08-20T17:06:58.153075-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7398,[phLogDetail]=631-DR, Step 1: check command type

2021-08-20T17:06:58.153099-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7407,[phLogDetail]=631-DR, Step 2: check command data

2021-08-20T17:06:58.153113-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7414,[phLogDetail]=631-DR, Step 3: load replication setting

2021-08-20T17:06:58.188916-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7444,[phLogDetail]=631-DR-init-primary, Step 1: Saving Disaster Recovery Settings on Primary

2021-08-20T17:06:58.217059-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7457,[phLogDetail]=631-DR-init-primary, Step 2: Saving peer SSH public key on Primary

2021-08-20T17:06:58.260626-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7471,[phLogDetail]=631-DR-init-primary, Step 3: Updating SSH Configuration on Primary

2021-08-20T17:06:58.306178-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7486,[phLogDetail]=631-DR-init-primary, Step 4: Updating SSH Known Hosts on Primary

2021-08-20T17:06:58.353953-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7508,[phLogDetail]=631-DR-init-primary, Step 5: Initializing Disaster Recovery

2021-08-20T17:06:58.422293-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7567,[phLogDetail]=631-DR-init-primary, Step 5.1: Running phinitprimary script on Primary

2021-08-20T17:06:59.039355-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7582,[phLogDetail]=631-DR-init-primary, Step 5.2: Sending Task to Secondary

2021-08-20T17:06:59.144875-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7619,[phLogDetail]=631-DR-init-primary, Step 5.3: Initializing Secondary

2021-08-20T17:06:59.188592-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 1

2021-08-20T17:06:59.220850-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 20

2021-08-20T17:06:59.250523-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 30

2021-08-20T17:06:59.284640-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 40

2021-08-20T17:06:59.310058-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 50

2021-08-20T17:09:34.909429-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 80

2021-08-20T17:09:34.987417-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7634,[phLogDetail]=631-DR-init-primary, Step 5.5: Secondary Setup complete

2021-08-20T17:09:34.987610-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7515,[phLogDetail]=631-DR-init-primary, Step 6: Waiting for App Server to come back up on Primary

2021-08-20T17:09:35.042360-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7523,[phLogDetail]=631-DR-init-primary, Step 7: Waiting for Secondary App Server to update Service Passwords

2021-08-20T17:09:35.106214-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=8187,[phLogDetail]=631-DR-init-primary, Step 7.1: get sevice user

2021-08-20T17:09:35.106241-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=8195,[phLogDetail]=631-DR-init-primary, Step 7.2: get secondary host

2021-08-20T17:09:35.106352-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=8214,[phLogDetail]=631-DR-init-primary, Step 7.3: send task updating Svc password to secondary

2021-08-20T17:09:35.180197-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=8224,[phLogDetail]=631-DR-init-primary, Step 7.4: finish updating Svc password on secondary

2021-08-20T17:09:35.182966-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7541,[phLogDetail]=631-DR-init-primary, Step 8: Restarting processes on Primary

2021-08-20T17:09:35.222909-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7549,[phLogDetail]=631-DR-init-primary, Step 9: Broadcasting DR tasks to all processes on Primary

2021-08-20T17:09:35.253335-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7559,[phLogDetail]=631-DR-init-primary, Step 10: Disaster Recovery setup complete

2021-08-20T17:09:35.276445-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7432,[phLogDetail]=631-DR, Step 4: initializing DR finished.

Failure to Connect to Secondary

If the message "Failed to connect secondary" appears, after enabling Disaster Recovery, try restarting phMonitor on the Secondary.

Backend Processes in DR Device are Down

If the backend processes in DR device are down and the response code of RESP APIs in /etc/httpd/logs/ssl_access_log is 401, then take the following steps.

  1. Get the new service password in the Primary server by running the following command:

    phLicenseTool --showServicePassword

  2. Send Rest API by Curl to the DR App server to update the password in the license file by running the following:

    curl -X PUT -u '1:<ServicePassword>' -H "Content-Type: text/xml" -d '<request><password>ServicePassword</password></request>' -k https://<DR_SUPER_IP>:443/phoenix/rest/custMgmt/service/changePwd

Troubleshooting Disaster Recovery

GUI

While Replication is being set up, you can see the progress in the GUI in two locations.

  1. ADMIN > License > Nodes > Add DR
    or

  2. Jobs/Errors > Jobs > filter on 6.3.1-DR

Backend Logs

On both the Primary and Secondary nodes, use the cat command to view the backend logs (/opt/phoenix/log/phoenix.log):

cat /opt/phoenix/config/phoenix.log

Successful Enablement of Disaster Recovery of the Primary node

2021-08-20T16:26:22.186396-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7398,[phLogDetail]=631-DR, Step 1: check command type

2021-08-20T16:26:22.186451-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7407,[phLogDetail]=631-DR, Step 2: check command data

2021-08-20T16:26:22.186475-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7414,[phLogDetail]=631-DR, Step 3: load replication setting

2021-08-20T16:26:22.222539-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7444,[phLogDetail]=631-DR-init-primary, Step 1: Saving Disaster Recovery Settings on Primary

2021-08-20T16:26:22.252896-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7457,[phLogDetail]=631-DR-init-primary, Step 2: Saving peer SSH public key on Primary

2021-08-20T16:26:22.291143-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7471,[phLogDetail]=631-DR-init-primary, Step 3: Updating SSH Configuration on Primary

2021-08-20T16:26:22.328987-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7486,[phLogDetail]=631-DR-init-primary, Step 4: Updating SSH Known Hosts on Primary

2021-08-20T16:26:22.503675-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7508,[phLogDetail]=631-DR-init-primary, Step 5: Initializing Disaster Recovery

2021-08-20T16:26:22.577513-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7567,[phLogDetail]=631-DR-init-primary, Step 5.1: Running phinitprimary script on Primary

2021-08-20T16:26:23.521939-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7582,[phLogDetail]=631-DR-init-primary, Step 5.2: Sending Task to Secondary

2021-08-20T16:26:23.579504-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7619,[phLogDetail]=631-DR-init-primary, Step 5.3: Initializing Secondary

2021-08-20T16:26:23.620795-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 1

2021-08-20T16:26:23.644367-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 20

2021-08-20T16:26:23.671393-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 30

2021-08-20T16:26:23.709927-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 40

2021-08-20T16:26:23.740996-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 50

2021-08-20T16:29:01.385371-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 80

2021-08-20T16:29:01.454035-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7634,[phLogDetail]=631-DR-init-primary, Step 5.5: Secondary Setup complete

2021-08-20T16:29:01.454250-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7515,[phLogDetail]=631-DR-init-primary, Step 6: Waiting for App Server to come back up on Primary

2021-08-20T16:29:01.509276-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7523,[phLogDetail]=631-DR-init-primary, Step 7: Waiting for Secondary App Server to update Service Passwords

2021-08-20T16:29:01.797224-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=8187,[phLogDetail]=631-DR-init-primary, Step 7.1: get sevice user

2021-08-20T16:29:01.797257-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=8195,[phLogDetail]=631-DR-init-primary, Step 7.2: get secondary host

2021-08-20T16:29:01.797440-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=8214,[phLogDetail]=631-DR-init-primary, Step 7.3: send task updating Svc password to secondary

2021-08-20T16:29:01.882229-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=8224,[phLogDetail]=631-DR-init-primary, Step 7.4: finish updating Svc password on secondary

2021-08-20T16:29:01.884157-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7541,[phLogDetail]=631-DR-init-primary, Step 8: Restarting processes on Primary

2021-08-20T16:29:01.923315-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7549,[phLogDetail]=631-DR-init-primary, Step 9: Broadcasting DR tasks to all processes on Primary

2021-08-20T16:29:01.962740-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7559,[phLogDetail]=631-DR-init-primary, Step 10: Disaster Recovery setup complete

2021-08-20T16:29:02.169745-07:00 va3005 phMonitorSupervisor[11908]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7432,[phLogDetail]=631-DR, Step 4: initializing DR finished.

2021-08-20T17:03:54.103262-07:00 va3005 phMonitorSupervisor[7384]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7712,[phLogDetail]=631-DR-init-secondary, Step 1: Initalize secondary From Primary server

2021-08-20T17:03:54.103449-07:00 va3005 phMonitorSupervisor[7384]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7733,[phLogDetail]=631-DR-init-secondary, Step 2: parse task info

2021-08-20T17:03:54.103689-07:00 va3005 phMonitorSupervisor[7384]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7757,[phLogDetail]=631-DR-init-secondary, Step 3: update SSH keys of primary on secondary

2021-08-20T17:03:54.115290-07:00 va3005 phMonitorSupervisor[7384]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7775,[phLogDetail]=631-DR-init-secondary, Step 4: update SSH configurations on secondary

2021-08-20T17:03:54.126466-07:00 va3005 phMonitorSupervisor[7384]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7793,[phLogDetail]=631-DR-init-secondary, Step 5: update SSH known hosts for primary on secondary

2021-08-20T17:03:54.148876-07:00 va3005 phMonitorSupervisor[7384]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7817,[phLogDetail]=631-DR-init-secondary, Step 6: run phinitsecondary script

2021-08-20T17:06:29.865977-07:00 va3005 phMonitorSupervisor[7384]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7881,[phLogDetail]=631-DR-init-secondary, Step 10: finish initialize secondary

Successful Enablement of Disaster Recovery of the Secondary node

2021-08-20T16:29:29.146469-07:00 andy-super-ipv6 phMonitorSupervisor[11936]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7712,[phLogDetail]=631-DR-init-secondary, Step 1: Initalize secondary From Primary server

2021-08-20T16:29:29.146829-07:00 andy-super-ipv6 phMonitorSupervisor[11936]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7733,[phLogDetail]=631-DR-init-secondary, Step 2: parse task info

2021-08-20T16:29:29.147537-07:00 andy-super-ipv6 phMonitorSupervisor[11936]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7757,[phLogDetail]=631-DR-init-secondary, Step 3: update SSH keys of primary on secondary

2021-08-20T16:29:29.160266-07:00 andy-super-ipv6 phMonitorSupervisor[11936]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7775,[phLogDetail]=631-DR-init-secondary, Step 4: update SSH configurations on secondary

2021-08-20T16:29:29.172964-07:00 andy-super-ipv6 phMonitorSupervisor[11936]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7793,[phLogDetail]=631-DR-init-secondary, Step 5: update SSH known hosts for primary on secondary

2021-08-20T16:29:29.287039-07:00 andy-super-ipv6 phMonitorSupervisor[11936]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7817,[phLogDetail]=631-DR-init-secondary, Step 6: run phinitsecondary script

2021-08-20T16:32:06.952364-07:00 andy-super-ipv6 phMonitorSupervisor[11936]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7881,[phLogDetail]=631-DR-init-secondary, Step 10: finish initialize secondary

2021-08-20T17:06:58.153075-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7398,[phLogDetail]=631-DR, Step 1: check command type

2021-08-20T17:06:58.153099-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7407,[phLogDetail]=631-DR, Step 2: check command data

2021-08-20T17:06:58.153113-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7414,[phLogDetail]=631-DR, Step 3: load replication setting

2021-08-20T17:06:58.188916-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7444,[phLogDetail]=631-DR-init-primary, Step 1: Saving Disaster Recovery Settings on Primary

2021-08-20T17:06:58.217059-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7457,[phLogDetail]=631-DR-init-primary, Step 2: Saving peer SSH public key on Primary

2021-08-20T17:06:58.260626-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7471,[phLogDetail]=631-DR-init-primary, Step 3: Updating SSH Configuration on Primary

2021-08-20T17:06:58.306178-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7486,[phLogDetail]=631-DR-init-primary, Step 4: Updating SSH Known Hosts on Primary

2021-08-20T17:06:58.353953-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7508,[phLogDetail]=631-DR-init-primary, Step 5: Initializing Disaster Recovery

2021-08-20T17:06:58.422293-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7567,[phLogDetail]=631-DR-init-primary, Step 5.1: Running phinitprimary script on Primary

2021-08-20T17:06:59.039355-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7582,[phLogDetail]=631-DR-init-primary, Step 5.2: Sending Task to Secondary

2021-08-20T17:06:59.144875-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7619,[phLogDetail]=631-DR-init-primary, Step 5.3: Initializing Secondary

2021-08-20T17:06:59.188592-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 1

2021-08-20T17:06:59.220850-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 20

2021-08-20T17:06:59.250523-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 30

2021-08-20T17:06:59.284640-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 40

2021-08-20T17:06:59.310058-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 50

2021-08-20T17:09:34.909429-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7629,[phLogDetail]=631-DR-init-primary, Step 5.4: Setting up Secondary: 80

2021-08-20T17:09:34.987417-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7634,[phLogDetail]=631-DR-init-primary, Step 5.5: Secondary Setup complete

2021-08-20T17:09:34.987610-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7515,[phLogDetail]=631-DR-init-primary, Step 6: Waiting for App Server to come back up on Primary

2021-08-20T17:09:35.042360-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7523,[phLogDetail]=631-DR-init-primary, Step 7: Waiting for Secondary App Server to update Service Passwords

2021-08-20T17:09:35.106214-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=8187,[phLogDetail]=631-DR-init-primary, Step 7.1: get sevice user

2021-08-20T17:09:35.106241-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=8195,[phLogDetail]=631-DR-init-primary, Step 7.2: get secondary host

2021-08-20T17:09:35.106352-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=8214,[phLogDetail]=631-DR-init-primary, Step 7.3: send task updating Svc password to secondary

2021-08-20T17:09:35.180197-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=8224,[phLogDetail]=631-DR-init-primary, Step 7.4: finish updating Svc password on secondary

2021-08-20T17:09:35.182966-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7541,[phLogDetail]=631-DR-init-primary, Step 8: Restarting processes on Primary

2021-08-20T17:09:35.222909-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7549,[phLogDetail]=631-DR-init-primary, Step 9: Broadcasting DR tasks to all processes on Primary

2021-08-20T17:09:35.253335-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7559,[phLogDetail]=631-DR-init-primary, Step 10: Disaster Recovery setup complete

2021-08-20T17:09:35.276445-07:00 andy-super-ipv6 phMonitorSupervisor[38314]: [PH_GENERIC_INFO]:[eventSeverity]=PHL_INFO,[procName]=phMonitorSupervisor,[fileName]=phMonitorProcess.cpp,[lineNumber]=7432,[phLogDetail]=631-DR, Step 4: initializing DR finished.

Failure to Connect to Secondary

If the message "Failed to connect secondary" appears, after enabling Disaster Recovery, try restarting phMonitor on the Secondary.

Backend Processes in DR Device are Down

If the backend processes in DR device are down and the response code of RESP APIs in /etc/httpd/logs/ssl_access_log is 401, then take the following steps.

  1. Get the new service password in the Primary server by running the following command:

    phLicenseTool --showServicePassword

  2. Send Rest API by Curl to the DR App server to update the password in the license file by running the following:

    curl -X PUT -u '1:<ServicePassword>' -H "Content-Type: text/xml" -d '<request><password>ServicePassword</password></request>' -k https://<DR_SUPER_IP>:443/phoenix/rest/custMgmt/service/changePwd