"Rube Goldberg" solutions

Rube Goldberg Machine

Faithful readers of my blog will know that normally my posts are about regulatory issues, but I thought it would be fun to take a break from the usual depressing posts about the state of telecom competition in Canada and the CRTC and talk about one of the more interesting problems a customer recently approached me to solve.

For those who don't know, my #dayjob is working as a consultant, providing customers with assistance designing and supporting their technology stacks - be it VoIP, cloud, networking, etc. The work is really interesting, but due to non-disclosure agreements I usually can't talk about what I'm doing or who I'm doing it for. Sometimes, however, customers give me the "all clear" to talk about a unique solution I developed for them, and so I figured I'd start a small series of blog posts talking about the more interesting problems and solutions I've designed.

The problem for this inaugural post was an interesting one - an executive user has a Logitech Rally Bar attached to a Samsung TV in their office for taking Microsoft Teams calls. When they are not in meetings, they want the TV to be showing a TV news feed on one of the other HDMI inputs. The challenge is that when the device rings, they don't always have time to find the remote to change the input back to the Rally Bar. So the ask was simple - have the Samsung TV change inputs from HDMI1 to HDMI2 whenever the user was called.

On the surface, this seemed simple, but as I dug into it, it became increasingly frustrating due to limitations in the Microsoft Teams APIs. Before I get into that, let's get into the easy part of problem - changing the TV input.

Step 1: Samsung SmartTV - Web Socket Control

Modern Samsung TVs (post 2016, mostly) running the Tizen OS have a very simple WebSocket protocol for interfacing with them. When you first connect an application to the TV, the user grants it authorization, and then the application can use the provided token to interface with the TV. Looking on github I found several existing libraries for controlling the TV, and doing a bit of reverse engineering built a very, very basic proof of concept Java client for Samsung SmartTVs.

Now that I could communicate with the TV and change the inputs as required, the next step was figuring out how to get notified that the user was being called.

Step 2: Get notified when the user was called

The first challenge I ran into with getting notified that the user was being called is that there is no existing Microsoft Graph API call for this functionality. In previous engagements working with APIs from other telephony vendors like Metaswitch and Broadsoft this was basic functionality, but not in the Teams ecosystem. Microsoft has graph API calls for a lot of things in Teams, but being notified when a user was being called wasn't one of them. There is an API to be notified when call detail records are created, but that only happens after the call has ended, so it didn't work for this use case.

Attempting to find a work-around I did a lot of googling and eventually stumbled across an interesting tool called "Teams Wizard" which claims to add the ability to run actions/commands on incoming calls, which is what I needed, but it only runs on Windows as an add-on to the desktop Teams client, so if the user didn't have their desktop signed in, I wouldn't get notifications. So while this tool may work for different use case I dismissed it and started looking for other solutions.

Going over the Microsoft documentation for Teams, I found there is one entity that can get notified via webhook when a call comes in - Teams Bots. The bot idea seemed intriguing and a viable path to getting the notification I needed, so I coded a basic Teams bot in Java and deployed it. After assigning my bot a Teams resource account and phone number via powershell, I called it and voila - I started getting HTTP POSTs from Teams when the user was called.

Step 3: Get my bot into the call path

Now that I had a basic working bot, the next question was how to get my bot into the call flow. I couldn't expect people to call the bot directly - that would break the normal teams workflow of clicking on a user and clicking call. What I needed was a way to keep the users call flow functionally the same but still get my bot in the flow. At first I thought about using the call delegates feature in Teams, but delegates can only be real users, bots can't be used. After mulling the problem for a bit the solution came to me - the Teams simultaneous ring feature. Even better was that support for controlling users call forwarding settings was recently added to Powershell, so I could add the bot to the users call flow without having to ask them to make changes in the Teams UI. I added my bot as a sim ring target to my phone and it worked exactly like I wanted.

Step 4: Get the notification to the TV

The final step was gluing it all together - I had a way to control the TV, I had my notification from Teams of the inbound call, I just needed to marry the two together.

Because the TV was located on a private network, and I didn't want to expose that network directly to the internet with ngrok or something I turned to using Azure Service Bus. I updated the bot to post a message to the service bus when the bot was called, then had a small Java app running on a Raspberry Pi inside the network subscribed to the queue. When the java app saw the inbound call message from the queue it invoked the Samsung API and changed the input. Great success!

Solution overview

So putting it all together, when a call comes in for the user, the Teams platform brings a Bot into the call flow via simultaneous ring. That Bot posts a message to the Azure message queue, which is then picked up by a application inside the network running on a Raspberry pi, which in turn changes the TV input via the Samsung API.

Overall, while complex, it's not the dirtiest solution I've ever had to come up with. Ideally Microsoft will eventually let applications subscribe to notifications for individual users and remove the need for the bot/sim ring kludge, but until then the customer is happy and they don't have to go scrambling for the TV remote every time someone calls them.

Hopefully you enjoyed this post - I'd love to hear any feedback or ideas on how you might have solved the same challenge.