Server Side Conversation History with SfB on premise and Exchange Online

Yes it works also in a hybrid scenario.

These two days I struggled to implement the new (well it dates from 2015) Server Side Conversation History for a customer who has a Skype for Business 2015 on-site (from Enterprise Voice and Telephony) and Exchange Online through Office 365.

So, I used some useful blog posts from Cloud Exchangers for the whole process understanding of Server Side Conversation History. I understood that for the Server Side Conversation History to work the cmdlet

Test-CsExStorageConnectivity –SipUri user@contoso.com -Verbose

has to pass successfully.

This command means that Oauth (Server to Server authentication) between SfB on prem and Exchange online should be OK.

So again, another blog post from Cloud Exchangers about OAuth is very useful to implement this.

However, at the end of the configuration, when the Test-CsExStorageConnectivity cmdlet it throws an error :

VERBOSE: Successfully opened a connection to storage service at localhost using binding: NetNamedPipe.
VERBOSE: Create message.
VERBOSE: Execute Exchange Storage Command.
VERBOSE: Processing web storage response for ExCreateItem Failure., result=ErrorUserSmtpMissing, activityId=2a173b88-ecb5-43ca-a0be-09f3fdb7bb2e, reason=StoreException: code=ErrorUserSmtpMissing, reason=Found user sipUri=[sip:user@contoso.com], but SmtpAddress is null/empty, ensure mailbox is enabled.
at
Microsoft.Rtc.Internal.Storage.Store.StoreConnectionManager.LookupUserDetails(StoreContext ctx, String sipUri, Guid& tenantId, String& smtpAddress, String&userSid,String& userUpn)
at
Microsoft.Rtc.Internal.Storage.Store.StoreConnectionManager.GetExchangeContext(StoreContext ctx, String sipUri)
at
Microsoft.Rtc.Internal.Storage.Store.StoreConnectionManager.GetExchangeClientProxy(StoreContext ctx, String sipUri, CacheMode cacheMode)
at
Microsoft.Rtc.Internal.Storage.Adaptor.ExStoreAdaptor.InternalCreateItem(StoreContext ctx, String sipAddress, CreateItemType createItem, Boolean autoCreateParentFolder, StoreAsyncResult`1 asyncResult, Boolean reAuthorize)
at
Microsoft.Rtc.Internal.Storage.Adaptor.ExStoreAdaptor.ExchangeCreateItem(StoreContext ctx, ExStoreRequest exStoreRequest, StoreAsyncResult`1 asyncResult)
at
Microsoft.Rtc.Internal.Storage.Adaptor.ExStoreAdaptor.BeginDispatchCommand(StoreContext ctx, StoreRequest request, AsyncCallback asyncCallback, Object state)
at
Microsoft.Rtc.Internal.Storage.Api.StorageService.BeginExecuteCommandInternal(Guid adapterId, StoreRequest request, AsyncCallback asyncCallback, Object state,
Boolean isAuthenticated)
.
VERBOSE: Activity tracing:
2016/10/19 11:57:48.179 Lookup user details, sipUri=sip:user@contoso.com, smtpAddress=, sid=, upn=user@contoso.com,
tenantId=00000000-0000-0000-0000-000000000000
VERBOSE: Unhandled response Microsoft.Rtc.Internal.Storage.StoreResponse. VERBOSE: Is command successful: False.
Test failed.

 

I double checked that the password for the User is the same between local Active Directory and Office 365. But I had no idea why am I receiving “SmtpAddress is null/empty, ensure mailbox is enabled” where the user has his mailbox fully activated and working on Exchange Online. (Remark : I have not used Dirsync, I just made sure that the password and SIP URI/EMAIL just match, I thought for a time not running Dirsync may be the problem but no… i worked without it, just read to the end)

I was desperate, especially when I read this topic on Microsoft Answers Community when somebody said it ain’t work with a such hybrid scenario.

Today, I checked another SfB deployment I have done for another customer but with Exchange on premise. I remarked that on Active Directory (open ADSI Edit), the ProxyAddresses field contains two entries which are “sip:user@contoso.com” and “smtp:user@contoso.com”.

I checked back to our problematic hybrid deployment, I found that the field ProxyAddresses contains ONLY “sip:user@contoso.com” but none smtp entry. It’s quite logic since we don’t have an Exchange On-Premise, we only have Exchange Online. And as I understood it’s while activating the user’s mailbox on Exchange that the SMTP entry is added to the ProxyAddresses field of the AD User.

So, you have understood everything !! What I have done is to manually add the SMTP field to the ProxyAddresses. Just open Powershell on Active Directory server and type this :

Set-ADUser -Identity your_user -Add @{ProxyAddresses=”SMTP:user@contoso.com”

And Guess what ? Got back to Skype for Business Management Shell to relaunch the Test-CsExStorageConnectivity cmdlet again and it showed “Test passed” ! It was because of the missing SMTP address in the ProxyAddresses field of the user located on Active Directory.

And Guess what again ? I just closed, removed all the cache, then opened Skype for Business on my Android and all the Conversation History is loaded ! IT WORKED!!

I have done the same procedure for another user having SfB on iPhone and it worked!

So the point is that integrating SfB on prem and Exchange online may have working from the beginning if we have, for example, an Hybrid Exchange shared between on premise and online. Because the Exchange on-prem should have already populated that SMTP entry in the ProxyAddresses field. But here we were on PURE Skype for Business on prem and PURE Exchange on Cloud.

And finally, the hybrid scenario of SfB on-prem with Exchange Online DO support Server Side Conversation History, not as it’s said in some forums.
I have a proof! I have a customer running Server Side Conversation History flawlessly with a such deployment scenario. Don’t be mistaken.

t38modem Windows binary HERE!! (+ Signed com0com drivers)

Today I’m giving you a very precious bounty.

Many people may already have t38modem windows binary but I really really struggled to get it. It’s like a rare material to find. All my compilation attempts of the project were big fails (VC++ 2005 and cross compilation and I don’t know what else… a nightmare!)

But when I found the t38modem binary, I thought everything is great. I went ahead but when I wanted to also use com0com, I realized that Windows doesn’t accept anymore unsigned  drivers. So com0com has to be signed. Some kind human being made that available for download.

So I bundled this and kept it in safe.
Don’t ask me for a specific version of t38modem, it’s version 1.1.0 it’s all what I have, nevermind for bugs etc. I’m sharing this as best effort.

It’s a full working FoIP windows binaries including t38modem and com0com.

You can use it for example with the built-in Windows Fax & Scan role in all Windows versions.

In my case, I also used a cron job on the Windows Server so that the .bat batch file (running t38modem with the relevant parameters) would run as soon as the machine comes up.

Enjoy!

The file name is : t38modem&com0com.zip

Download it here from 4shared.

Its MD5 hash is : 0dcb573daeb1a9d924a6243f11f3b1fa

Why you should not upgrade your WebEx to 2.7

… Until you retire the last computer using Windows 7 in your company, or even until your last customer stop using it.

So here we are.

Webex development team decided to drop support of TLS 1.0 for security reasons.

Here’s an excerpt from the CWMS 2.7 Release Notes :

TLS Support

CWMS Release 2.7 supports TLS 1.1 and later; TLS 1.0 is not supported, with one exception. Client connections to an SMTP server using TLS 1.0 are supported.

And as I far as I’m concerned, Windows 10 general adoption is still so far from being a reality. Moreover, as far as I know, Windows 7 is not yet (at least today) End of Support.

Since that by default, Windows 7 Internet Options only support TLS 1.0 and earlier versions of SSL. (unless you check TLS 1.1 and TLS 1.2), a default common configuration of Windows 7 won’t be able to connect to a WebEx meeting hosted on CWMS 2.7

Given that changing a such company-wide parameter is not easy at all, and that since Webex is not only meant for internal users, but also for guest and external users (that we need to give the easiest possible webconferening user experience and are more likely to have Windows 7 than any other OS), I think that Cisco should have simply said “Windows 7 is not supported with CWMS 2.7”. It would be quite simpler and more alarming for administrators who attempts to upgrade their CWMS System (2.6 to 2.7 shouldn’t be that difference)

I have seen many other situations where Cisco dropped support for OS/Browsers for far less limitations that this.

Yes, I’m not happy. I’m going to fresh reinstall CWMS 2.6 tomorrow after an entire week to deploy CWMS 2.7 😦

 

UCCX Agent reporting outage frenzy

CUIC and HRC don’t generate Agent reports

Monday, september 12 2016. Yes we all love mondays very much hein!!

My phone rang from 3 different customers.

“Hello! We have a problem on CUIC reports! We can’t generate reports!!”

“Good morning! We are unable to generate our daily Agent Summary report! We can’t count the working hours of our employees! We need this for calculating wages”

“Hello, when I try to generate an Agent Report on Historical Reporting Client, we get an error, I sent it to you by email, check it out :
Nested SQLException; SQLState: IX000 Vendor code: -1265 Message: Overflow occurred on a datetime or interval operation. ”

Well the statements above are sorted from worst to best customer (I’m sure you understand why when you read the 3 different problem descriptions). We all have a customer that doesn’t give any further information about incidents!!! Just “we have a problem, come onsite”, as dry as that.

(Please if you are customer be like the 3rd and not 1st one)

 

Anyway, the problem is general and covers all versions from 8.x to latest 11.x

One Cisco developer has been coding UCCX Reporting anyhow and declared the date variable in a horrible manner. As a result all CCX deployments suffered that 12 Sep from an outage in Agent Reports. In the Bug Search toolkit, I have seen the number of cases is around 425 after only two days, good luck TAC people!

Luckily Cisco published a bug fix called ciscouccx.ReportFix.cop.sgn and it immediately solves the issue. No restart/reboot of services/server is needed. Don’t forget to install it on both nodes not the Publisher only.
You can download the COP file from this supportforums topic.

Otherwise, contact TAC so they send it to you (hey! You don’t like what I said YOU who refuses to renew Service Contrat! Haha. This time it’s OK there’s a COP file, tomorrow who say what happens ? Long life to Cisco Products.)

 

 

EDIT : The COP file is no longer shared in the supportforums. You have to download it directly from Cisco and yes, you need a valid support contract to do so.

7800-8800 series IP phones reject old MD5 hashed CUCM certificates

Result : Extension Mobility not working on 7800/8800 series IP phones

Last week I have been running into an issue regarding Extension Mobility for a customer who upgraded his UC platform from 6.1 to 11.5
His platform contains, inter alia, one CUCM Publisher and one CUCM Subscriber

By this occasion he expanded his platform with a bunch of new 7821 IP phones.

The upgrade ran flawlessly and everything works for the existing 7900 serie IP phones.

However, when adding 7821 IP phones we realized that there’s a problem with Extension Mobility. They get “Host not found” (the most famous error for Phone Services).

They are using the Secure URL

https://Publisher_IP_Address:8443/emapp/EMAppServlet?device=#DEVICENAME#

With this service, all 7900 IP phones work, but all 7821 don’t.
Moreover, from the logs and Wireshark traces, we don’t even see the IP phones sending requests to TVS on port 2445. Niente. Here is what the log said :

2373 ERR Sep 05 12:29:41.264147 (2092:2355) JAVA-Sec SSL Connection – Handshake failed.

 

We added a second EM service to further troubleshoot the issue, this time with the Subscriber IP address. Here, all IP phones including 7821 phones run EM correctly. What the hell ?

After two weeks of troubleshooting with TAC, we took the two tomcat certificates and compared them. SURPRISE!! The Publisher’s tomcat certificate is Version 1 and uses the MD5 hash algorithm!! WOW quite old fashion (whereas the Subscriber is Version 3 and uses SHA256).

But why ? It’s because of the well-kept CUCM 6.1 certificate!
My preferred method of upgrading (from MCS to UCS) is to backup/restore only the Publisher and then fresh reinstall the subsequent Subscribers.
In fact, CUCM 6.1 generates its certificate with the above attributes.

Meanwhile, the CUCM Subcriber tomcat certificate is good because it was indeed freshly installed.
As soon as we regenerated the tomcat Publisher certificate (this time with more “modern” attributes) then restarted Cisco Tomcat service on Publisher, the Extension Mobility works flawlessly on all IP phones including 7800/8800.

What’s the point ?
7800/8800 IP phones don’t support certificates running with MD5 hash algorithm. YES they will reject it. It’s the new law of newer Cisco IP phones. (but the quite Cisco 7900 serie Phones accept this cert without any concern).

So I come to my personal conclusion. When you upgrade CUCM from very old versions to 11.5, and using default self-signed certs, it’s a MUST to regenerate tomcat certificate otherwise Extension Mobility on 7800/8800 series IP phones won’t work.
It’s not said anywhere in the upgrade guides/documentation, but I learnt it the hard way.

bad-good-certs

Bonus : this also affects UDS (User Data Services) in Jabber.
Jabber on Windows doesn’t like that certificate either. It gives an error – Unable to verify certificate or something like that.
Regenerating the tomcat cert on Publisher also resolves this issue.

Hello World!

Aloha from Tunisia!

Gonna present myself. I’m a Voice & Unified Communications consultant and I’m from Tunisia (the country whose flag is like Turkey’s one). This is my very first post.

I have been working for 4 years so far with Cisco UC technologies with liiiiiiiiiiiiittle bit of Lync/SfB. No, I didn’t choose my job. It’s the last thing I thought of when graduating. Working on “Voice & UC”, Wat!! Sorry man I don’t understand Chinese.
In fact, I didn’t even hear about this technology before I actually began work. Yet I’m loving it.

My feelings about my work ? A rollercoaster of obsession and happiness!
You guess that, all depends on how things are turning on customers’ site.

Today, I have decided more than ever before to write a blog (yes, I thought about it since long time but now enough!!) and OK let it be “Yet Another Unified Comm blog”. At least I won’t hide my obsessions (ahh Bugs!!) and then happiness of finding the solution.

Good reading and thank you for your visit.