Unable To Recreate Data Guard Fast Start Failover Configuration With DGMGRL (Doc ID 454418.1)

https://support.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=359626911642608&id=454418.1&displayIndex=1&_afrWindowMode=0&_adf.ctrl-state=ki2nqstjf_471#FIX

Unable To Recreate Data Guard Fast Start Failover Configuration With DGMGRL (Doc ID 454418.1)

To Bottom

---

In this Document

Symptoms

Cause

Solution

---

Applies to:

Oracle Database - Enterprise Edition - Version 10.2.0.1 to 12.1.0.2 [Release 10.2 to 12.1]
Information in this document applies to any platform.
***Checked for relevance on 21-Sep-2012***
*** Reviewed for Relevance 16-Jul-2015 ***

Symptoms

-- Problem Statement:
On 10.2.0.x in Production:

When attempting to recreate a Data Guard Broker Configuration the following Error occurs when attempting
to edit the Configuration Property LogXptMode in the Primary Site :

ERROR

DGMGRL> edit database "ORCL1" set property LogXptMode='SYNC';

Error: ORA-16654: Fast-Start Failover is enabled.

-- Steps To Reproduce:
This Issue can be reproduced at will with the following Steps:

Remove the Data Guard Broker Configuration while Fast Start Failover is enabled.

1. Set up a Fast Start Failover Configuration and ensure FS_FAILOVER_STATUS is SYNCHRONIZED.

2. Place the Standby Database into READ ONLY Mode and shutdown the Standby Listener.

3. Perform a DML load on the Primary until FS_FAILOVER_STATUS becomes UNSYNCHRONIZED

4. Once the UNSYNCHRONISED State can be seen in the Primary site, kill the DMON-Process and remove the DG Broker Configuration Files.

5. DMON will restart but the FS_FAILOVER_STATUS will remain UNSYNCHRONISED.

6. At this Point the Creation of a new Data Guard Broker Configuration fails when attempting to edit the
configuration Property LogXptMode and set it to SYNC.

-- Business Impact:
The Issue has the following Business Impact:

Due to this Issue, the Data Guard Broker Configuration cannot be re-enabled and the Standby Database is
no longer protected by the Data Guard Broker and the Observer. So Fast-Start Failover is not possible at this Stage.

Cause

It appears that the Standby Protection Level of MaxAvailability is preventing the enabling of a
new Data Guard Broker Configuration. The original Configuration was removed incorrectly and is resulting
in the Failure to enable a new Configuration.

The Data Guard Physical Standby Fast Start Failover was found to be in a state of UNSYNCHRONIZED.
At this point the Data Guard Broker Configuration Files were removed from the OS Level and the DMON
Processes on each Node in the Standby Configuration were killed using OS Commands to respawn new
Processes and release the deleted Configuration Files.

Each Time the Primary Site Data Guard Broker Configuration is edited and the Property LogXptMode is set to SYNC it fails reporting Error:

ORA-16654: Fast-Start Failover is enabled.

For example:

DGMGRL> edit database "ORCL1" set property LogXptMode='SYNC';

Error: ORA-16654: Fast-Start Failover is enabled.

At this point in time in the Primary Database the Standby Database Protection Mode is set to
Maxavailability.

SQL> select DB_UNIQUE_NAME,FORCE_LOGGING,FLASHBACK_ON,LOG_MODE,OPEN_MODE,DATABASE_ROLE, GUARD_STATUS,PROTECTION_MODE,FS_FAILOVER_STATUS
from v$database

DB_UNIQUE_NAME FOR FLASHBACK_ON LOG_MODE OPEN_MODE DATABASE_ROLE
-------------- --- ------------- ------------ ---------- ----------------
GUARD_S PROTECTION_MODE FS_FAILOVER_STATUS
------- -------------------- ---------------------
orcl1 YES YES ARCHIVELOG READ WRITE PRIMARY
NONE MAXIMUM AVAILABILITY UNSYNCHRONIZED

Solution

-- To implement the solution, please execute the following steps (see Commands below)::

1. Clean up the Primary and Standby Sites.

2. Disable the Data guard Broker on each Instance

3. Remove the Data Guard Broker Configuration Files using OS-Tools.

4. Downgrade to Maximum Performance Mode on the Primary Site

5. Enable the Data Guard Broker again on all Instances

6. Recreate the Data Guard Broker Configuration

7. Raise Protection Mode back to maximum Availability using the Data Guard Broker

8. Enable Fast Start Failover again.

For example:

At the primary site:

SQL> alter system set dg_broker_start=false scope=spfile;

System altered.

SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.

SQL> ! rm $ORACLE_HOME/dbs/dr*.dat

SQL> startup nomount
ORACLE instance started.

Total System Global Area 314572800 bytes
Fixed Size 1261516 bytes
Variable Size 281018420 bytes
Database Buffers 25165824 bytes
Redo Buffers 7127040 bytes

SQL> alter system set dg_broker_start=true;

System altered.

SQL> ! ls -l $ORACLE_HOME/dbs/dr*.dat
ls: /opt/app/oracle/product/10.2.0/db_2/dbs/dr*.dat: No such file or directory

SQL> alter database mount;

Database altered.

SQL> alter database set standby database to maximize performance;

Database altered.

SQL> select DB_UNIQUE_NAME,FORCE_LOGGING,FLASHBACK_ON,LOG_MODE,OPEN_MODE,DATABASE_ROLE, GUARD_STATUS,PROTECTION_MODE,FS_FAILOVER_STATUS
from v$database

DB_UNIQUE_NAME FOR FLASHBACK_ON LOG_MODE OPEN_MODE DATABASE_ROLE
-------------- --- ------------- ------------ ---------- ----------------
GUARD_S PROTECTION_MODE FS_FAILOVER_STATUS
------- -------------------- ---------------------
orcl1 YES YES ARCHIVELOG MOUNTED PRIMARY
NONE MAXIMUM PERFORMANCE DISABLED


SQL> alter database open;

Database altered.

SQL> select DB_UNIQUE_NAME,FORCE_LOGGING,FLASHBACK_ON,LOG_MODE,OPEN_MODE,DATABASE_ROLE, GUARD_STATUS,PROTECTION_MODE,FS_FAILOVER_STATUS
from v$database

DB_UNIQUE_NAME FOR FLASHBACK_ON LOG_MODE OPEN_MODE DATABASE_ROLE
-------------- --- ------------- ------------ ---------- ----------------
GUARD_S PROTECTION_MODE FS_FAILOVER_STATUS
------- -------------------- ---------------------
orcl1 YES YES ARCHIVELOG READ WRITE PRIMARY
NONE MAXIMUM PERFORMANCE DISABLED

At the standby site:

SQL> alter system set dg_broker_start=false scope=spfile;

System altered.

SQL> shutdown immediate
ORA-01109: database not open


Database dismounted.
ORACLE instance shut down.

SQL> ! rm $ORACLE_HOME/dbs/dr*.dat

SQL> startup nomount
ORACLE instance started.

Total System Global Area 314572800 bytes
Fixed Size 1261516 bytes
Variable Size 247463988 bytes
Database Buffers 58720256 bytes
Redo Buffers 7127040 bytes

SQL> alter system set dg_broker_start=true;

System altered.

SQL> ! ls -l $ORACLE_HOME/dbs/dr*.dat
ls: /opt/app/oracle/product/10.2.0/db_2/dbs/dr*.dat: No such file or directory

SQL> alter database mount;

Database altered.

SQL> select DB_UNIQUE_NAME,FORCE_LOGGING,FLASHBACK_ON,LOG_MODE,OPEN_MODE,DATABASE_ROLE, GUARD_STATUS,PROTECTION_MODE,FS_FAILOVER_STATUS
from v$database

DB_UNIQUE_NAME FOR FLASHBACK_ON LOG_MODE OPEN_MODE DATABASE_ROLE
-------------- --- ------------- ------------ ---------- ----------------
GUARD_S PROTECTION_MODE FS_FAILOVER_STATUS
------- -------------------- ---------------------
orcl1stb YES YES ARCHIVELOG MOUNTED PHYSICAL STANDBY
NONE MAXIMUM PERFORMANCE DISABLED


From the dataguard command line:

DGMGRL> connect /
Connected.
DGMGRL> show configuration;
Error: ORA-16532: Data Guard broker configuration does not exist

Configuration details cannot be determined by DGMGRL

DGMGRL> create configuration ORCL1DG as primary database is 'ORCL1' connect identifier is
'ORCL1_ELLE';
Configuration "orcl1dg" created with primary database "ORCL1"

DGMGRL> add database 'ORCL1STB' as connect identifier is 'ORCL1_HALLE' maintained as physical;
Database "ORCL1STB" added

DGMGRL> show configuration;

Configuration
Name: orcl1dg
Enabled: NO
Protection Mode: MaxPerformance
Fast-Start Failover: DISABLED
Databases:
ORCL1 - Primary database
ORCL1STB - Physical standby database

Current status for "orcl1dg":
DISABLED

DGMGRL> enable configuration;
Enabled.

DGMGRL> show configuration;

Configuration
Name: orcl1dg
Enabled: YES
Protection Mode: MaxPerformance
Fast-Start Failover: DISABLED
Databases:
ORCL1 - Primary database
ORCL1STB - Physical standby database

Current status for "orcl1dg":
SUCCESS

DGMGRL> edit database "ORCL1" set property LogXptMode='SYNC';
Property "logxptmode" updated

DGMGRL> edit database "ORCL1STB" set property LogXptMode='SYNC';
Property "logxptmode" updated

DGMGRL> EDIT CONFIGURATION SET PROTECTION MODE AS MaxAvailability;
Succeeded.

DGMGRL> enable fast_start failover;
Enabled.

DGMGRL> show configuration;

Configuration
Name: orcl1dg
Enabled: YES
Protection Mode: MaxAvailability
Fast-Start Failover: ENABLED
Databases:
ORCL1 - Primary database
ORCL1STB - Physical standby database
- Fast-Start Failover target

Current status for "orcl1dg":
Warning: ORA-16608: one or more databases have warnings


From the Dataguard Observer node:

[oracle@elle ~]$ dgmgrl -logfile $HOME/observer.log sys/oracle@orcl1_elle "start observer" &
[1] 25249

[oracle@elle ~]$ DGMGRL for Linux: Version 10.2.0.3.0 - Production

Copyright (c) 2000, 2005, Oracle. All rights reserved.

Welcome to DGMGRL, type "help" for information.
Connected.

[oracle@elle ~]$ ps -ef | grep dgmgrl
oracle 24421 13170 0 13:34 pts/3 00:00:00 dgmgrl
oracle 25249 24675 0 13:36 pts/1 00:00:00 dgmgrl -logfile
/home/oracle/observer.log start observer
oracle 25330 24675 0 13:37 pts/1 00:00:00 grep dgmgrl


From the dataguard command line:

DGMGRL> show configuration;

Configuration
Name: orcl1dg
Enabled: YES
Protection Mode: MaxAvailability
Fast-Start Failover: ENABLED
Databases:
ORCL1 - Primary database
ORCL1STB - Physical standby database
- Fast-Start Failover target

Current status for "orcl1dg":
SUCCESS