How to change weblogic password from Backend in EBS 12.2.x?

On EBS 12.2.X environments , if you are on R12.AD.C.Delta.7 and R12.TXK.C.Delta.7 or later, we can change Weblogic admin server password using the below script.
1. Startup admin server. Do not start any other services
$. ./EBSapps.env RUN
$ cd $ADMIN_SCRIPTS_HOME
$adadminsrvctl.sh start
2. Run the below script  
perl $FND_TOP/patch/115/bin/txkUpdateEBSDomain.pl -action=updateAdminPassword
[The script will prompt for Current and New Weblogic Password and restarts Weblogic admin server using the new password.]
3. Run fs_clone to propagate the changes to the alternate file system (Patch File system).
$adop phase=fs_clone

How to Update APPS password in EBS Datasource (EBS 12.2.x)?

 From EBS 12.2.x versions, whenever we change apps password we need to update the new apps password in EBS Datasource.
Otherwise managed servers wont come up.
Steps to update apps password in Weblogic datasource for EBS 12.2x, post AD-TXK 7:
1. Start Only Admin server,Do not start any other services
$. ./EBSapps.env RUN
$ cd $ADMIN_SCRIPTS_HOME
$adadminsrvctl.sh start

2. Update apps password in Datasource from backend:
$perl $FND_TOP/patch/115/bin/txkManageDBConnectionPool.pl
When prompted select the “updateDSPassword” option.
Sample screen output:

perl $FND_TOP/patch/115/bin/txkManageDBConnectionPool.pl
Please select from list of valid options
        updateDSPassword – Update WebLogic Datasource Password
        updateDSJdbcUrl  – Update WebLogic Datasource Connection String
Enter Your Choice : updateDSPassword
Enter the full path of Applications Context File [DEFAULT -]:
Enter weblogic admin server password:
Enter the APPS user password:

Note: We can also update the apps password in EBS Datasource from Weblogic console. 

Concurrent request Error: The executable file for this concurrent program cannot be executed.

 Issue: Concurrent request completed with below Error.

This request finished with an error and produced the following completion message: The executable file /test1/apps/fs2/EBSapps/appl/fnd/12.0.0/bin/XXCUST_PROG for this concurrent program cannot be executed.

ERROR:

The executable file /test1/apps/fs2/EBSapps/appl/fnd/12.0.0/bin/XXCUST_PROG for this concurrent program cannot be executed.

Contact your system administrator or support representative.

Cause:

Verify that the execution path to the executable file

Run the below command

$ls -lrt /test1/apps/fs2/EBSapps/appl/fnd/12.0.0/bin/XXCUST_PROG

Output of the above command shows executable is pointing to a wrong symbolic link


Solution:

1.Create a soft link with the program name as below:

$cd /test1/apps/fs2/EBSapps/appl/fnd/12.0.0/bin/
$ln -s -f $FND_TOP/bin/fndcpesr XXCUST_PROG


2. Re-run the concurrent program and test the issue.

How to update override email address using backend sql query in EBS 11i/12.x?

Query to check Override address in EBS workflow notification Mailer:

SQL>
 select fscpv.parameter_value
    from fnd_svc_comp_params_tl fscpt
    ,fnd_svc_comp_param_vals fscpv
    where fscpt.display_name = ‘Test Address’
    and fscpt.parameter_id = fscpv.parameter_id;   

PARAMETER_VALUE
——————————————————————————–
NONE

SQL Query to update Override email address from backend in EBS :

SQL> update fnd_svc_comp_param_vals
set    parameter_value = ‘TEST_EMAIL@DOMAIN.COM’
where  parameter_id =
( select parameter_id
 from   fnd_svc_comp_params_tl
 where  display_name = ‘Test Address’);  

1 row updated.

SQL> commit;

Commit complete.


Verify that override email address is updated:

SQL> select fscpv.parameter_value
    from fnd_svc_comp_params_tl fscpt
    ,fnd_svc_comp_param_vals fscpv
    where fscpt.display_name = ‘Test Address’
    and fscpt.parameter_id = fscpv.parameter_id;  
PARAMETER_VALUE
——————————————————————————–
TEST_EMAIL@DOMAIN.COM

SQL>

EBSO 12.2.x : Query to check status of ADOP session

Check Status of adop session using below sql query:

set pagesize 200;
set linesize 160;
col PREPARE_STATUS format a15
col APPLY_STATUS format a15
col CUTOVER_STATUS format a15
col ABORT_STATUS format a15
col STATUS format a15
select NODE_NAME,ADOP_SESSION_ID, PREPARE_STATUS , APPLY_STATUS  ,CUTOVER_STATUS , CLEANUP_STATUS , ABORT_STATUS , STATUS
from AD_ADOP_SESSIONS
order by ADOP_SESSION_ID;

Decoding Cutover status:



cutover_status=’Y’ ‘COMPLETED’

cutover_status not in (‘N’,’Y’,’X’) and status=’F’ ‘FAILED’
cutover_status=’0′ ‘CUTOVER STARTED’
cutover_status=’1′ ‘SERVICES SHUTDOWN COMPLETED’
cutover_status=’3′ ‘DB CUTOVER COMPLETED’
cutover_status=’D’ ‘FLIP SNAPSHOTS COMPLETED’
cutover_status=’4′ ‘FS CUTOVER COMPLETED’
cutover_status=’5′ ‘ADMIN STARTUP COMPLETED’
cutover_status=’6′ ‘SERVICES STARTUP COMPLETED’
cutover_status=’N’ ‘NOT STARTED’
cutover_status=’X’ ‘NOT APPLICABLE’

EBS 12.2.X Patching: ADOP Cutover failed while running txkADOPCutOverPhaseCtrlScript.pl

Issue:

 During an adop online patching cycle, cutover phase failed while running the script txkADOPCutOverPhaseCtrlScript.pl


  [PROCEDURE]  Starting Middle Tier Services
    [UNEXPECTED]Error occurred running “perl /test/app/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPCutOverPhaseCtrlScript.pl -contextfile=/test/app/fs1/inst/apps/EBSTEST_testebsmt111/appl/admin/EBSTEST_testebsmt111.xml -patchcontextfile=/test/app/fs1/2/inst/apps/EBSTEST_testebsmt111/appl/admin/EBSTEST_testebsmt111.xml -promptmsg=hide -console=off -sessionid=14 -timestamp=20201108_194155 -outdir=/test/app/fs_ne/EBSapps/log/adop/14/20201108_194155/cutover/testebsmt111 -action=forcestartup”
    [UNEXPECTED]Cutover phase has failed.

Cause: 

Review of all adop and cutover log files shows that Cutover phase failed after filesystem switch and ADMIN server startup,oacore services could not be started after cutover due to port conflict.

Resolution:

1. Check Status of adop session using below sql query:
set pagesize 200;
set linesize 160;
col PREPARE_STATUS format a15
col APPLY_STATUS format a15
col CUTOVER_STATUS format a15
col ABORT_STATUS format a15
col STATUS format a15
select NODE_NAME,ADOP_SESSION_ID, PREPARE_STATUS , APPLY_STATUS  ,CUTOVER_STATUS , CLEANUP_STATUS , ABORT_STATUS , STATUS
from AD_ADOP_SESSIONS
order by ADOP_SESSION_ID;

For my issue, cutover status was 5, which means ADMIN startup is completed.

2. Restart the cutover phase with mtrestart=no option

adop phase=cutover mtrestart=no


3. Fix the issues related to services startup on RUN filesystem (which is switched after cutover).

4. Startup the services

5. Perform cleanup phase and complete the adop cycle

adop phase=cleanup 

Useful srvctl commands

Useful srvctl commands for ORACLE RAC MAINTENANCE

srvctl -help or srvctl -v


srvctl commands to startup database:

srvctl start database -d db_name [-o start_options] [-c connect_str|-q]
srvctl start database -d db_name [-o open]
srvctl start database -d db_name -o nomount
srvctl start database -d db_name -o mount

srvctl commands to shutdown database:

srvctl stop database -d db_name [-o stop_options] [-c connect_str|-q]
srvctl stop database -d db_name [-o normal]
srvctl stop database -d db_name -o transactional
srvctl stop database -d db_name -o immediate
srvctl stop database -d db_name -o abort


srvctl config database
srvctl config database -d db_name [-a] [-t]


srvctl start listener -n node_name [-l listener_names]
srvctl stop listener -n node_name [-l listener_names]


srvctl status listener [-n node_name] [-l listener_names] 
srvctl config listener -n node_name

adpreclone.pl Error on Applications Tier

Issue:
adpreclone.pl appsTier on EBS 12.2.x environment fails with error

ERROR: The context variable s_apps_jdbc_connect_descriptor is null

Cause:

Value of parameter “s_apps_jdbc_connect_descriptor” is null in Applications Context file.

Solution:

1. Take Backup of context file

2. Update the value of parameter “s_apps_jdbc_connect_descriptor” with JDBC connect string to connect to the database.

3. Run Autoconfig on Applications node.

4. Restart adpreclone.pl appsTier