Cloning - Apps DBA Diaries

ADOP fs_clone failing with error : “Could not find patch context file from database”

Issue:

FS_CLONE is failing with error :Could not find patch context file from database
Error details:
$adop phase=fs_clone
==============
Checking for existing adop sessions.
No pending session exists.
Starting new adop session.
===========================================================================
ADOP (C.Delta.12)
Session ID: 43
Node: test1.domain1.com
Phase: fs_clone
Log: /test/erp/apps/fs_ne/EBSapps/log/adop/32/20220121_145748/adop.log
===========================================================================
Verifying existence of context files in database.
[UNEXPECTED] Could not find patch context file from database
[UNEXPECTED]Patch edition context file not found in database for host test1.domain1.com

Summary report for current adop session:
Node test1.domain1.com:
– Fs_clone status: Not Started
For more details, run the command: adop -status -detail

[STATEMENT] Please run adopscanlog utility, using the command
“adopscanlog -latest=yes”
to get the list of the log files along with snippet of the error message corresponding to each log file.

adop exiting with status = 2 (Fail)

=========================
Cause:
===
Patch context file not found in fnd_oam_context_files tables.

Solution:
=======
1.Source the patch file system and run autoconfig on patch filesystem to update context file in database.
or
2. Run CtxSynchronizer to upload patch context file to database.

$ $ADJVAPRG oracle.apps.ad.autoconfig.oam.CtxSynchronizer action=upload contextfile=<patch context file name and location> logfile=/tmp/patchctxupload.log

Provide apps password when prompted.

“APP-FND-02704: Unable to alter user APPS_NE to change password” while changing APPS password using FNDCPASS

Issue:

Changing apps password using FNDCPASS fails with below error

APP-FND-02704: Unable to alter user APPS_NE to change password.
Oracle error 28003: has been detected in alterpassword2.

Command to change APPS Password.[Ensure to shut down Application services before changing apps password]
FNDCPASS apps/<apps_pwd> 0 Y system/<password> SYSTEM APPLSYS <new_apps_password>
Error Message from Logfile:
+—————————————————————————+
Application Object Library: Version : 12.2
Copyright (c) 1998, 2013, Oracle and/or its affiliates. All rights reserved.
FNDCPASS:
+—————————————————————————+
Current system time is 10-APR-2020 18:43:36
+—————————————————————————+
Arguments
FNDCPASS system/***** SYSTEM APPLSYS *****
+—————————————————————————-+
Working…
APP-FND-02704: Unable to alter user APPS_NE to change password.
Oracle error 28003: has been detected in alterpassword2.
+—————————————————————————+
Concurrent request completed
Current system time is 10-APR-2020 18:43:39
+—————————————————————————+

Cause:
Issue is due to Password_verify_function enables in dba_profiles for APPS,APPLSYS,APPS_NE users

Solution:

Disable the password verify function for APPS,APPLSYS,APPS_NE users in database profiles.
1. Identify the database profile for APPS,APPLSYS,APPS_NE users using below query.

SELECT USERNAME,PROFILE FROM DBA_USERS where USERNAME in (‘APPS’,’APPLSYS’,’APPS_NE’);
SELECT PROFILE,VALUE FROM DBA_PROFILES WHERE PROFILE=’&profile for user’;

2. Disable password verify function

alter profile <profile_name> limit PASSWORD_VERIFY_FUNCTION null;

3. Run the FNDCPASS command to change password.

FNDCPASS apps/<apps_pwd> 0 Y system/<password> SYSTEM APPLSYS <new_apps_password>

4. After changing apps password, run autoconfig on Middle tier and update EBS datasource for R12.2.x

You can follow the below steps to update apps password in EBS Datasource:

http://www.appsdbadiaries.com/2022/02/how-to-update-apps-password-in-ebs.html

5. Startup Application services

4. Revert back the changes done in step 2

How to change weblogic password from Backend in EBS 12.2.x?

On EBS 12.2.X environments , if you are on R12.AD.C.Delta.7 and R12.TXK.C.Delta.7 or later, we can change Weblogic admin server password using the below script.

1. Startup admin server. Do not start any other services

$. ./EBSapps.env RUN
$ cd $ADMIN_SCRIPTS_HOME
$adadminsrvctl.sh start

2. Run the below script

perl $FND_TOP/patch/115/bin/txkUpdateEBSDomain.pl -action=updateAdminPassword

[The script will prompt for Current and New Weblogic Password and restarts Weblogic admin server using the new password.]

3. Run fs_clone to propagate the changes to the alternate file system (Patch File system).

$adop phase=fs_clone

How to Update APPS password in EBS Datasource (EBS 12.2.x)?

From EBS 12.2.x versions, whenever we change apps password we need to update the new apps password in EBS Datasource.
Otherwise managed servers wont come up.
Steps to update apps password in Weblogic datasource for EBS 12.2x, post AD-TXK 7:
1. Start Only Admin server,Do not start any other services
$. ./EBSapps.env RUN
$ cd $ADMIN_SCRIPTS_HOME
$adadminsrvctl.sh start

2. Update apps password in Datasource from backend:
$perl $FND_TOP/patch/115/bin/txkManageDBConnectionPool.pl
When prompted select the “updateDSPassword” option.
Sample screen output:

perl $FND_TOP/patch/115/bin/txkManageDBConnectionPool.pl
Please select from list of valid options
updateDSPassword – Update WebLogic Datasource Password
updateDSJdbcUrl – Update WebLogic Datasource Connection String
Enter Your Choice : updateDSPassword
Enter the full path of Applications Context File [DEFAULT -]:
Enter weblogic admin server password:
Enter the APPS user password:

Note: We can also update the apps password in EBS Datasource from Weblogic console.

adpreclone.pl Error on Applications Tier

Issue:

adpreclone.pl appsTier on EBS 12.2.x environment fails with error

ERROR: The context variable s_apps_jdbc_connect_descriptor is null

Cause:

Value of parameter “s_apps_jdbc_connect_descriptor” is null in Applications Context file.

Solution:

1. Take Backup of context file

2. Update the value of parameter “s_apps_jdbc_connect_descriptor” with JDBC connect string to connect to the database.

3. Run Autoconfig on Applications node.

4. Restart adpreclone.pl appsTier

Autoconfig Error : ORA-01422: exact fetch returns more than requested number of rows

Issue: Autoconfig on EBS 12.2 environment fails while running script txkGenADOPWrapper.pl

Below is the error message from logfile

=======================

Script Name : txkGenADOPWrapper.pl

Script Version : 120.0.12020000.3

ERROR DESCRIPTION:

(*******FATAL ERROR*******

PROGRAM : (/test/app/fs1/inst/apps/test_mtnode1/admin/install/txkGenADOPWrapper.pl)

TIME : Wed Nov 25 04:22:34 2020

FUNCTION: TXK::SQLPLUS::_doExecute [ Level 3 ]

MESSAGES:

SQLPLUS error: buffer=

SQL*Plus: Release 10.1.0.5.0 – Production on Wed Nov 25 04:22:33 2020

SQL> SQL> Connected.

SQL> SELECT ad_zd_adop.get_node_type(‘mtnode1’) FROM DUAL

ERROR at line 1:

ORA-01422: exact fetch returns more than requested number of rows

ORA-06512: at “APPS.AD_ZD_ADOP”, line 3010

=======================

Cause:

FND_NODES table has incorrect node information.

Found an extra row in FND_NODES table where node_name is null after clone.

Solution:

1. Clean up FND_NODES table

sqlplus apps/<password>

sql>exec FND_CONC_CLONE.SETUP_CLEAN;

2.Run Autoconfig on Database node(s).

cd $ORACLE_HOME/appsutil/bin

adconfig.sh

[Provide Database Context file and apps Password]

3. Run Autoconfig on apps tier

Source the RUN filesystem environment file.

cd $AD_TOP/bin

adconfig.sh

[Provide Applictions Context file and apps Password]

RMAN-03009: failure of allocate command on ORA_AUX_SBT_TAPE_1 channel

This is an error during Database Clone using RMAN. I was trying to restore database using RMAN Backup

RMAN-00571: ===========================================================

RMAN-00569: ===============
ERROR MESSAGE STACK FOLLOWS ===============

RMAN-00571: ===========================================================

RMAN-03002: failure of Duplicate Db command
RMAN-05501: aborting duplication of target database

RMAN-03009: failure of allocate command on ORA_AUX_SBT_TAPE_1 channel
ORA-19554: error allocating device, device type: SBT_TAPE, device name:

ORA-27211: Failed to load Media Management Library

Additional information:

Cause: Backup Configuration was modified.

Solution:
Use the below command to Resolve the error and continue with Database Restore

rman auxiliary /
run
{
allocate auxiliary channel c1 type disk;
allocate auxiliary channel c2 type disk;
duplicate database to ‘DEV’ backup location ‘/Daily_Backup/Backup_16042018’;
}
exit;
EOF

adcfgclone.pl script failed during instantiate OHS Stage

While cloning 12.2.6 environment, adcfgclone.pl script failed during instantiate OHS Stage.

Issue is because opmn components were unable to start up with below error messages.

globalInitNLS: NLS boot file not found or invalid
— default linked-in boot block used
XML parser init: error 201.
globalInitNLS: NLS boot file not found or invalid
— default linked-in boot block used

Further analysis on the clone logfiles for OHS creation shown that ORA_NLS10 parameter is unset before cloning.

Resolution:

$unset ORA_NLS10

+Restart the failed script