UCM Default Certificate Renew

[Alfordap]

I have a CS1000E
Type: Nortel Communication Server 1000E CPPM
Version: 4021
Release: 700 Q +
The current default ssl certificate is expiring. The private Cert Authority is still good so I figured I would just go into the UCM-Security-Certificates and renew it. All the manuals say that you just "Click the Default Certificate Link" and follow the steps. The link is not clickable. Another tech here vaguely remembers having to run a command in the linux CLI to allow the default certificated to be changed but doesnt know the command. I have looked through the Admin and Security manual and cannot find any such command. I have also looked at the harden command help and nothing that stands out there either. Anyone have any ideas?

 

[Firebird Scrambler]

I found this report below that might be related.

Problem
CS1000 rls 7.65: unable to access Element Manager from UCM

Answer
check/correct system platform time
check system manager open ssl x509 cert from CLI
unable to open EM from UCM elements page
X509 certifificate error observed.

FIX
Login to the smgr via ssh and su - root
Then type: openssl x509 -in /var/lib/pgsql/data/server.crt -noout -enddate
Shows something like this:
notAfter=April 15 08:42:49 2015 GMT

in this case the certificates were valid

from CLI checked the DATE and the Time was incorrect

accessed System Platform Web console to correct the Time and restarted System Platform

accessed System Manager CLI to restart the Jboss 'service jboss restart'

From UCM access the EM experiencing X509 issue, Web GUI now displays at this point still no access
from web GUI sign in as local login from front page
You will then need to run through the Primary Security Configuration info then Restart the application

on doing this was then able to access the EM via the UCM.





Firebird Scrambler

Nortel & Avaya Meridian 1 / Succession & BCM / Norstar Programmer

Website =

http://somertel.net

linkedin

 

[Alfordap]

I found that one also but I that was for a valid certificate and the time on the server was wrong. I actually have a certificate that is going out of date and need to replace the certificate.

 

[Firebird Scrambler]

I've found some other references below

Problem
Unable to access UCM services from Avaya Aura System Manager.
Also unable to login to CS1000

Answer
Postgresql certificate expired.
The Postgresql log (/var/lib/pgsql/data/pg_log/) shows the following:-

LOG: could not accept SSL connection: sslv3 alert certificate unknown
LOG: could not accept SSL connection: sslv3 alert certificate unknown

Postgresql certificate expired.

All commands need to be run in root.

To check the certifcate expiry date:-
openssl x509 -in /var/lib/pgsql/data/server.crt -noout -enddate

if the certificate has expired it can be renewed by running the following scripts

/opt/Avaya/Postgres/9.0.2/utils/securePostgres.sh

Recheck the certificate:-

openssl x509 -text -in /var/lib/pgsql/data/server.crt|grep 'Not After'

THEN REBOOT THE SERVER
(It might look as if it is ok on first look without the reboot, but feedabck from site says other things don't work)





##############################################

Problem
Unable to login to Avaya System Manager Web pages - white page displayed only

Answer
Check for expired postgres certificates

Details:
Customer reporting unable to login to system manager with any of their user accounts. System appears to take user credentials and then on pushing the enter key nothing happens except for a white page to be displayed.
There is an Avaya documented problem where by as part of the certificate renewal process the postgres cert does not get updated and as a result the white page is displayed upon login.
Fix:
Login to the smgr via ssh and su - root
Then type: openssl x509 -in /var/lib/pgsql/data/server.crt -noout -enddate
Shows something like this:
notAfter=April 15 08:42:49 2015 GMT
Then depending upon which version of SMGR you have locate the appropriate directory
SMGR 5.2
/opt/Avaya/Postgres/8.3.7/utils/securePostgres.sh
SMGR 6.0.x / 6.1.x
/opt/Avaya/Postgres/8.4.4/utils/securePostgres.sh
SMGR 6.2.x
/opt/Avaya/Postgres/9.0.2/utils/securePostgres.sh
SMGR 6.3.x
/opt/Avaya/Postgres/9.1.3/utils/securePostgres.sh
SMGR 6.3.9 to 6.3.13 (Fixed in SP14)
/opt/Avaya/Postgres/9.2.4/utils/securePostgres.sh
Once you have identified you version of Postgres then type the following command replacing the example postgres with your appropriate version

# sh /opt/Avaya/Postgres/8.4.4/utils/securePostgres.sh

Stopping postgresql service: [ OK ]

Starting postgresql service: [ OK ]
Then type:
# openssl x509 -text -in /var/lib/pgsql/data/server.crt|grep 'Not After'
(Now showing a date in the future)
Not After : Apr 15 11:11:29 2017 GMT


NOTE: if postgres certificate was renewed this way. UCM components may fail to appear in the webconsole. Reboot SMGR before final verification.

Running the above script is not service effecting but it’s wise to make sure that no one is trying accessing the System Manager (web page or CLI) during this operation. It takes not more than 1 or 2 mins to complete the process.

Make sure to restart jboss: service jboss restart
Wait 10 mins then attempt to successfully login again to the SMGR web page withe the user account credentials

##################################

Problem
System Manager Geographic replication between the primary and secondary show as disabled and will not re-enable

Answer
Details:
System Manager Geographic replication disabled and will not re-enable

Fix:
After taking an smgr backup from the primary system manager server I did all of the following

1. Checked the geoRedundancyOperational.log on the primary server located at var/log/Avaya/mgmt/geo to try and gain information about the failure & see things like below

Feb 4 18:18:22 auramanager.scotamb.co.uk GeoUI: +00:00 2015 250 1 com.avaya.mgmt | 1 com.avaya.mgmt.geo.replication GEOREPL00085 Error resuming streaming replication; cause cannot resume; need fullsync
<11>Feb 11 11:09:55 auramanager.scotamb.co.uk GeoUI: +00:00 2015 588 1 com.avaya.mgmt | 0 com.avaya.mgmt.geo.replication.service.LDAPReplicator backup called, but this is a PRIMARY server in the DISABLE workflow, so no backup is being made.

2.Converted the primary to a standalone.

3. Re-run the GEOGRAPHIC Redundancy configuration on the secondary & observed the following in the autoReconfig.log located at /home/ucmdeploy/quantum
.
Thu Dec 10 12:37:22 GMT 2015 SMGR :: AutoConfig file not consumed or errored.
Thu Dec 10 12:37:22 GMT 2015 SMGR :: operationStatus=failed
Thu Dec 10 12:37:22 GMT 2015 SMGR :: ErrorMessage=Automated configuration has failed

4. So at this point we know that the reconfiguration to primary and secondary SMGR servers had failed and a rollback to two standalone servers had taken place

5.Ran a validateSMGR on both servers to see if I could see anything obvious on both servers. Location /var/vsp to find the script to run at root level

6 Navigate to opt/Avaya/Mgmt/6.3.8/rts/update/scripts & look for the update scripts by doing an ll and you will see something like
rw-r--r-- 1 admin admin 1443 Oct 3 2014 06021200_06030100.sh
-rw-r--r-- 1 admin admin 742 Oct 3 2014 06030300_06030400.sh

Then run both the scripts with command: sh 06030300_06030400.sh
You will see something like this:

CREATE FUNCTION
upgrade_manageabilitystatus
-----------------------------
0
(1 row)

DROP FUNCTION
SUCCESS: Executed sql functions successfully.
]0;root@auramanager:/opt/Avaya/Mgmt/6.3.8/rts/update/scripts[root@auramanager scripts]# sh 06021200_06030100.sh
UPDATE 1
UPDATE 1
UPDATE 1
SUCCESS: updated rts successfully.


7.Checked the identity certificates on both servers and found the mismatch between the 2 year validity period, so forced a renewal of both, from the cli of both servers
(use the CertificateRenewalUtility.bin file for this, using the force command ,when running the script) command = sh CertificateRenewalUtility.bin -f

8 Restart both SMGR templates from the cli using smgr restart & wait for the restart to complete (10 mins approx).

9.Re-run the GEO configuration on the secondary server whilst observing the autoReconfig.log file located at: /home/ucmdeploy/quantum Type: tail -f autoReconfig.log

Example:
Thu Dec 10 16:22:35 GMT 2015 SMGR ::
Thu Dec 10 16:22:35 GMT 2015 SMGR :: Stopping JBoss...
Perform cleanup ....
Stopping System Manager JBoss - PID: 27452
.............................................
Stopped System Manager JBoss
Thu Dec 10 16:24:06 GMT 2015 SMGR ::
Thu Dec 10 16:24:06 GMT 2015 SMGR :: Quantum Un-Configuration...
Thu Dec 10 16:25:07 GMT 2015 SMGR :: Un-Configuring UDDI Component...
Thu Dec 10 16:25:07 GMT 2015 SMGR :: Postgres status check
(pid 27130) is running...
Thu Dec 10 16:25:07 GMT 2015 SMGR :: Postgres is started
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'business_category'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'business_descr'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'business_identifier'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'business_name'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'business_service'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'contact'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'discovery_url'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'address'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'binding_template'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'contact_descr'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'email'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'phone'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'service_category'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'service_descr'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'service_name'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'address_line'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'binding_category'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'binding_descr'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'tmodel_instance_info'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'instance_details_descr'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'instance_details_doc_descr'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:1: NOTICE: truncate cascades to table 'tmodel_instance_info_descr'
TRUNCATE TABLE
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:3: NOTICE: truncate cascades to table 'tmodel_category'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:3: NOTICE: truncate cascades to table 'tmodel_descr'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:3: NOTICE: truncate cascades to table 'tmodel_doc_descr'
psql:/home/ucmdeploy/quantum/juddi_clean_db.sql:3: NOTICE: truncate cascades to table 'tmodel_identifier'
TRUNCATE TABLE
TRUNCATE TABLE
UPDATE 8
UPDATE 16
UPDATE 1
UPDATE 8
UPDATE 8
UPDATE 0
DELETE 0
DELETE 1
Thu Dec 10 16:25:07 GMT 2015 SMGR :: UDDI Component Un-Configuration Complete...
Thu Dec 10 16:25:07 GMT 2015 CND ::
Stopping slapd: [ OK ]

Thu Dec 10 16:25:07 GMT 2015 CND :: Installing CND...
dos2unix: converting file /etc/rc.d/init.d/cnd to UNIX format ...
Shutting down kernel logger: [ OK ]

Shutting down system logger: [ OK ]

Starting system logger: [ OK ]

Starting kernel logger: [ OK ]

[ OK ]
Starting slapd: [ OK ]
Thu Dec 10 16:25:14 GMT 2015 CND ::
Thu Dec 10 16:25:14 GMT 2015 CND :: CND Un-Configuration completed... successfully
Stopping slapd: [ OK ]

Thu Dec 10 16:25:14 GMT 2015 SMGR :: Setting Quantum DNS server ...
Thu Dec 10 16:25:14 GMT 2015 SMGR :: Quantum DNS server already exists.
Thu Dec 10 16:25:14 GMT 2015 SMGR ::
Thu Dec 10 16:25:14 GMT 2015 SMGR :: Quantum Un-Configuration completed... successfully
Thu Dec 10 16:25:14 GMT 2015 SMGR :: Quantum successfully unconfigured.
Thu Dec 10 16:25:14 GMT 2015 SMGR :: Starting Quantum Auto Configuration Prepare
Thu Dec 10 16:25:14 GMT 2015 SMGR :: Using supplied autoConfig.properties to configure Quantum as a secondary.
Thu Dec 10 16:25:14 GMT 2015 SMGR :: Completed Quantum Auto Configuration Prepare
Thu Dec 10 16:25:14 GMT 2015 SMGR :: Quantum auto config successfully prepared.
Thu Dec 10 16:25:14 GMT 2015 SMGR :: Reset Ownerships...
Thu Dec 10 16:25:14 GMT 2015 SMGR :: Effective user is: root
Thu Dec 10 16:25:14 GMT 2015 SMGR :: Confirm that 'admin' user exists...
uid=783(admin) gid=783(admin) groups=783(admin),700(nortel)
Setup Linux Base deployment environment.
groupadd: group nortel exists
useradd: user nortel exists
Thu Dec 10 16:25:46 GMT 2015 SMGR :: Reset Ownerships... completed
Thu Dec 10 16:25:46 GMT 2015 SMGR :: Setting file permissions complete.
Thu Dec 10 16:25:46 GMT 2015 SMGR :: Starting JBOSS to trigger Quantum auto-configuration.
Thu Dec 10 16:33:29 GMT 2015 SMGR :: :20: Quantum Auto Configuration still running.
Thu Dec 10 16:33:59 GMT 2015 SMGR :: :19: Quantum Auto Configuration still running.
Thu Dec 10 16:34:29 GMT 2015 SMGR :: Quantum autoConfig file successfully consumed.
Thu Dec 10 16:34:48 GMT 2015 SMGR :: JBoss started successfully
Thu Dec 10 16:34:48 GMT 2015 SMGR :: Performing final JBoss restart to finish reconfiguration.
Thu Dec 10 16:47:10 GMT 2015 SMGR :: JBoss started successfully
Thu Dec 10 16:47:10 GMT 2015 SMGR :: operationStatus=success
Thu Dec 10 16:47:10 GMT 2015 SMGR :: Quantum has been successfully configured as a secondary

After all of the services have restarted, login to both servers and verify that one now shows as a Primary SMGR and the other shows as a Secondary

Then on the nominated Primary, the enable replication button should now have changed from 'greyed out' as per the beginning of the issue, to being available to activate, click the button and after a few seconds you should be able to observe the replication down to the secondary should taking place.

The time taken for this to complete was approximately 10 mins but I believe that this can vary due to size of database bandwidth etc

Once complete it should show a successful completion with the date and time showing on the Primary SMGR dashboard. This same information will also be propagated across to the
Secondary system manager and should show as identical on that GR dashboard as well









Firebird Scrambler

Nortel & Avaya Meridian 1 / Succession & BCM / Norstar Programmer

Website =

http://somertel.net

linkedin

 

[Alfordap]

Firebird, I appreciate it but most of this is for Aura not CS1000E 7.0

 

[Firebird Scrambler]

It's all that I can find. Sorry.

Firebird Scrambler

Nortel & Avaya Meridian 1 / Succession & BCM / Norstar Programmer

Website =

http://somertel.net

linkedin

 

[Alfordap]

Turns out that I could install a new WebSSL cert and after a restart of Jboss with a new WebSSL cert installed it overrides the default certificate and the problem is fixed.

 

[leekawai]

@Alfordap,
How to install the new WebSSL cert ? Would you give us more info /howto ? Thx very much

 

[Alfordap]

leekawai
I was able to renew it in the UCM.
Under "Certificates" you can select which endpoint address you want to manage. This will show you all the certificates.
If you click on "Web SSL" you will get a pop up window that gives you the following options

Export a certificate and its private key to a file​

Replace the current certificate​

Remove the current certificate​

Create a certificate renew request​

The signing authority on my UCM was still good so I used the "Replace the current certificate" option and followed the instructions in the window.
If your certificates are externally signed then use the "Create a certificate renew request" option and follow the on screen instructions
After you follow the instructions to get the certificate replaced you will need to restart Jboss from the UCM command line. The command is appstart Jboss-Quantum restart