A year and a half ago, I heeded the growing warning signs that indicated the looming demise of macOS, née OS X, as a platform for developer and true computer enthusiasts, and set about trying to find a new ecosystem. Luckily, this was around the same time that updates to Windows 10 combined with the the continued awesomeness of WSL,1 made it a viable option for reconsideration.
While Windows had always remained my primary development platform of choice due to the nature of my work, there were various apps that I’d become accustomed to using and had to seek out alternatives for, chief of which was iMessage. As someone that has never embraced the mobile craze, I sorely missed the ability of texting (or “iMessaging”) from my PC, and came to absolutely despise having to drag my phone out of my pocket and text from its cramped display, constantly fighting autoincorrect and embarrassing myself with typos and misspellings. I tried switching to alternative platforms, including WhatsApp, Skype, and FB Messenger; but nothing came close to matching the pure simplicity and sheer genius of iMessage’s “SMS backwards-compatibility” approach that upgrades iPhone-to-iPhone communications to iMessage while transparently falling back to SMS or MMS where iMessage was not an option.
After a certain point, my frustration at living without iMessage reached the point where I was entertaining dangerous thoughts of trading in my Precision 5520 for an Emoji MacBook “Pro” when I decided to take matters into my own hands and set off to reverse engineer iMessage, wondering how hard it could be to use my old 15″ rMBP as an iMessage gateway. I was certainly not the first to entertain such thoughts, and online search results were cluttered with jetsam and flotsam from such failed or abandoned attempts in the almost 10 years since iMessage was first launched as an iPhone-only exclusive.
iMessage isn’t exactly an open platform, and Apple has gone to quite the extent to obscure the details of its inner workings. While parts of the protocol have been reverse-engineered and documented over time, the actual implementation has continued to grow and mutate to encompass more features and enhance privacy and security. There’s absolutely no sanctioned API for interfacing with iMessage programmatically (which has helped to curtail the expansion of spammers on the platform), and Apple has actually gone to certain undocumented extents to stealthily block the use of iMessage even on seemingly genuine OS X/macOS installations.2
Unlike web-based platforms such as WhatsApp and Facebook Messenger that operate over fairly standard RESTful services across a variety of different hardware and software combinations making them more amenable to replication and easier reverse-engineering, iMessage has the distinct strategic advantage of operating in a very small subset of devices, and is implemented in native code, making it all the harder to intercept.3 Apple’s renewed focus on security has also made it harder to piggyback onto the iMessage network, and it was fairly clear that trying to create a standalone iMessage client on Windows without going through either an iPhone or Mac as an iMessage proxy would be at best extremely difficult and highly fragile – but mostly just a futile waste of time.
It seemed at the start that the best bet would be to focus on the (extremely limited) surface area of iMessage functionality exposed to AppleScript, which could be used – with some awkwardness and lots of guesswork – to at least implement some basic message forwarding capabilities. Lots of trial-and-error and digging through Google/InternetArchive cached copies of long-gone web pages resulted in some success when it came to extending this approach to support limited outgoing messaging to existing chats, but it soon became apparent that Apple had gone out of their way to quite intentionally and very secretively4 break even what limited functionality the AppleScript approach offered.
Disappointed with the crippled functionality that relying on AppleScript to send and receive iMessages offered, I decided to try my hand at interfacing directly with the same libraries iChat/Messages.app offered as a last attempt before installing OS X Mountain Lion (the first OS X to feature iMessage support, prior to the intentional crippling of iMessage via AppleScript) on my rMBP and resigning myself to running that for the foreseeable future. While I had never written more than a few lines (consisting of mainly C code interfacing with Carbon, at that) of Objective C, I knew that iOS developers had for years circumvented certain App Store restrictions by using private frameworks intended for exclusive consumption by first-party Apple apps, and presuming that something as complicated as iMessage available on both mobile and desktop platforms and integrated heavily throughout both macOS and iOS would certainly have a framework that exposed its functionality. While there wasn’t a lot of material available online when it came to reverse-engineering private framework on macOS, there was however no shortage of similar resources for iOS developers, helpful even in their abandoned or even broken state.
For someone that had never developed a modern iOS/macOS application (despite having more than a decade of experience with C and C++), there were a lot of false starts trying to figure out just where this well-hidden functionality that would allow one to achieve the holy grail of iMessage: the ability to send an arbitrary message to any number of recipients on or off of the iMessage network – previously contacted or otherwise – with that elusive blue bubble. In the process, I discovered countless bugs in the iMessage implementation on iOS and macOS, and came across numerous traps and pitfalls. Some of these could be chalked up to genuine oversight where Apple developers did not expect a certain API to be (ab)used in a certain way and the client on one or more of iOS or macOS would react in unexpected ways (crashing, losing message history, etc) but others were so bizarre and so random that the only conclusion that could be reached was that they were part of the same campaign that Apple had embarked upon to prevent developers from hacking their way into the iMessage network.
For instance, it took almost no time at all to find a basic API in one of the private frameworks used by Messages.app that appeared to be exactly what I was looking for: a function that would send an
NSString to an address specified by another
NSString – sounds perfect! Giddy with excitement, hardly daring to believe it could be as simple as this, I gave it a try and decided to text my VoIP work number. A few seconds later, my notifications app pinged to let me know that an incoming SMS had made its way, from me, to me, programmatically generated in XCode! Any thoughts I may have had about this being virtually over and done with quickly came crashing down, however, when I tried to send an iMessage to myself, from my phone number to my iMessage-associated email address.
It turns I lucked out the first time in that I picked an SMS recipient, without iMessage support. Whatever address I messaged via this API on my MacBook resulted in an outgoing SMS to the address/number in question, valid or otherwise. This included email addresses, too, as I found out when I “successfully” received a message from myself via email – it turned out macOS responded to my API call by generating a text message with my iCloud email as the destination address, and Verizon responded by generating an email with their SMS-to-email gateway and obligingly sending it my way.
Other attempts resulted in the exact opposite experience. I found out that internally macOS determines which network to send a message on via the
imessage:// destination prefix (bizarrely, even when specifying
onService:@"iMessage" in specific API calls), but try as I might, I couldn’t figure out a way to determine a priori if a given phone number was on the iMessage network or not – at least, not with the APIs that I’d discovered by dumping
MessagesKit and other private frameworks.
At one point, I thought I’d had it beat. I was able to use the private messaging APIs to create – but, importantly, not send – a new conversation in Messages.app; which I could then take advantage of from another API to determine the nature of the destination address, relying on the fact that iMessage identifies (via a remote ping) whether a message to a target address would go out as an SMS or as an iMessage as soon as you type in the address, before any message is ever composed or sent (that’s how it decides whether to show the contact’s name in green or blue, depending on whether the contact is reachable over the iMessage network or not).
I hastily fired off a number of messages, repeating a routine I’d memorized and come to despise, testing each of the following single-recipient scenarios:
- Messages to myself over SMS,
- Messages to myself over iMessage (from one email address associated with my iMessage account to another email address associated with the same account, a trick that always works to send yourself a note),
- Messages to a number known to myself but not to iMessage5 to be an SMS-only target,
- Messages to a number similarly known only to myself to be iMessage-capable,
- Messages to an email address not on the iMessage network.
I held my breath and waited for the resounding ping letting me know the message was received on each of the target devices, not daring to believe it could – and yet, at this point, the expression “too good to be true” no longer held any meaning. After all, it had been many, many months since I’d first embarked on this journey.
My first endeavour in the Fall of 2015 at keeping iMessage whilst cutting macOS out of my life was a straightforward enough job – one I’d never dreamed would ultimately take on the life it did, spanning many generations of completely different mechanisms across more than six months of on-again/off-again attempts during which I oscillated in an almost bipolar fashion between the extreme euphoria that came with the satisfaction of thinking I’d finally begun to figure things out and the feelings of intense hopelessness and despair as the inevitable realization broke that I’d wasted countless days and nights toiling over something fundamentally broken in a giant game of cat-and-mouse, up against the endless resources and might of the Cupertino giant.
That first attempt relied on an AppleScript fired off via the basic “execute AppleScript on message received” and “execute AppleScript on message sent” hooks exposed in the Messages.app preferences on macOS (repeatedly broken in macOS updates and now completely removed) that I’d coded to execute a simple utility with the sender, recipient, and body as command line arguments. That utility would, in turn, compose an email addressed to myself and a few milliseconds later, my Windows PC would chime with a notification bubble from
email@example.com letting me know that my wife had safely made it to her destination on a snowy day or that so-and-so was late to a meeting I’d forgotten was even taking place.
While it addressed the most pressing of my problems (namely, how can I skim & sift through incoming messages or reference a past message quickly without having to deal with the cumbersome task of context switching to my mobile device) — and, not to mention, meant I no longer had to deal with that horrid on-screen keyboard, cramped display, or the pathetic excuse for search that using my iPhone entailed — it just wasn’t enough.
I felt that if I were to be intellectually honest with myself, a read-only incoming message notification API, while decent, was really just a poor imitation that deserved to be put to a quick and merciful death rather than continue to parade about claiming to be
firstname.lastname@example.org when it really should have been named email@example.com. It wasn’t a point of pride or a true accomplishment, just a reminder at how well Apple had succeeded at tethering me to their platform – and a throwback to 2010 when I finally, reluctantly agreed to part ways with my BlackBerry Curve 8900 with its beautiful, precise keyboard and reliable BBM for that gorgeous, to-die-for retina display on the iPhone 4.
It was then that my first “real” attempt at actually replacing iMessage began. Once I’d ascertained that it would be possible to get some form of iMessage proxying working for both incoming and outgoing messages – even if the latter would be extremely limited and constrained – I set about simultaneously working on a better UX than my email inbox. The first step was to port the existing SMTP-based message-forwarding code to an interface that would better reflect the nature of conversations, allow bi-directional communication, and provide multipresence so that I could use it at work and at home on my many PCs, laptops, and workstations.
I’m ashamed to say that my first attempt involved swapping out email for Slack. In my own defense, I’d deployed a few open source XMPP servers and toyed with the (extremely few) various multipresence extensions that were available (in fact, the choice of XMPP server depended on the quality of the multipresence plugins/integration available for it), but ultimately was extremely disappointed in the functionality offered by the XMPP servers (message history, multipresence, etc) and the clients (high-dpi support, multipresence and message history compatibility, attachment support, etc). What Slack was offering was really enticing: a simple-to-use API, search, multipresence complete with read message syncing, and (ugly, bloated, and slow as it was) a client that worked on Windows 10 and took advantage of at least some of its features (high-dpi support, notification toasts, etc).
That experiment didn’t last too long. It began with the discomfort that gnawed at me from the start and only grew with each incoming message as I considered the fact that I’d just taken what was more-or-less a fairly secure platform and – willfully and knowingly, no less! – shared my contacts, my conversations, my private affairs with Slack, a company I knew nothing about and couldn’t possibly ever trust. My reservations about continuing to rely on Slack as my frontend to a future “iMessage for Windows” only intensified as my interactions with the client grew. Quite apart from the bloat and distinct “uncanny valley” that Slack-the-Electron-app brought as a single-page webapp bundled in a resource-heavy standalone Chrome instance, famed for bringing down lesser PCs and workstations across Silicon Valley and the world at large, I was growing extremely disappointed with the clumsiness of the Slack UX itself. It was clear that dealing with more than a few “conversations” at a time was not something the team over at Slack HQ had given much thought to: the strict, static ordering of conversations in the sidebar, the restrictions on group names, the difficulty of switching between rooms, and more grew only more apparent with each new conversation, and I found myself actively avoiding messaging people if only because it meant having to dig through the conversation list and try to identify the phone number associated with their conversation.6
I wanted a client that offered everything Slack did and more. A client that looked at least somewhat aesthetically clean instead of embracing the most hideous shade of aubergine possible,7 a client that didn’t suck up an insane amount of RAM or need to establish a connection just to show me my messages. A client that sorted conversations/rooms reverse chronologically, that sync’d read messages back to my iPhone and MacBook. A client that showed me the names of people I was texting and integrated with my contacts. A client that let me associate multiple addresses with one contact. A client that let me rename contacts as they got married or as I learned their last names.8 A client that didn’t store my private data on a remote server out of my control. A modern client for a modern OS that felt like the real thing. I wanted iMessage for Windows, and “iMessage for Slack” wasn’t it.
None of my Slack woes really mattered if I couldn’t get the remaining problems with the iMessage API sorted out. What good is a native iMessage client for Windows 10 if it can only send to people I’ve messaged before – and one that only received iMessages and not SMSs at that!9 It was back to the drawing board for me.
We left off on the server side of things with an attempt that appeared to work, correctly distinguishing between iMessage and SMS for even unknown contacts, that correctly sent to both numbers and email addresses without need for manual user intervention. Well, long story short, that particular test didn’t work. I thought it had, Messages.app told me it had, looking at my iPhone appeared to confirm it had – the messages had all gone out in various shades of green and blue… but they never actually reached. I’d somehow tricked all my iDevices into thinking a new conversation had been started and messages had been sent, but it was all a cruel hoax played on me by the ghost of Steve Jobs, who gleefully tittered away, pleased at the success the roadblocks set up at his direction had in stymying my efforts at ditching macOS but taking iMessage with me.
It took some more work and many more sleepless nights, but I actually did finally figure out how to get outgoing messages to work. It was sheer, dumb luck, actually. I’d sloppily copied-and-pasted one of my tests involving a function exposed by a private framework and ran
make test. I realized just after I hit the Enter key that I’d forgotten to correct one of the parameters.. except my phone was vibrating and the previously blank terminal window pinned to a corner of my desktop was suddenly, impossibly, displaying the contents of the message I’d sent. It didn’t make sense, but then again, it was clearly never meant to. It had worked.
At this point, invigorated by my success and determined not to go down another rabbit hole or waste my time implementing a full-fledged iMessage client only to later discover that a critical feature was missing from the iMessage proxy server, I decided to fully flesh out the API. Piecing together the various components I’d uncovered or invented over the past six months, I slowly but surely built up the iMessage proxy API. All but one key component, of course.
I’m not sure if it’s just that lady luck was particularly generous when it came to stifling developers’ fledgling attempts at interfacing with iMessage or if it was especially creative greybeards hiding out in their cubicles, ignoring the world and cackling aloud to themselves at the thought of an innocent developer just trying to scratch an itch for the sake of intellectual satisfaction and curiosity with a particular need to achieve feature parity perfection, but there was one iOS messaging feature that eluded me no matter how long I chased it or which approach I tried. Group messaging, despite the quickly discovered presence of APIs that perfectly paralleled those for sending a message to a single individual, was a particularly onerous undertaking, and one that I just couldn’t get it to work.
At this point we come across the most nasty of the dead ends and booby traps left in place, hinted at before. Prior to this, as I’d mentioned, I’d encountered numerous “edge cases” of iMessaging behavior: messages that weren’t sent, messages that purportedly failed but resulted in confused replies “why did you send me this message forty times?” as my scripts responded to a particular (false!) error code by retrying the transmission, and other bugs of that nature that are only to be expected when trying to coerce and cajole an API not intended for 3rd party consumption to do your bidding. But this one was an ugly doozy, and not in very good form – if only there were someone with whom I could lodge a complaint.
You see, apart from a case I ran into where messaging a group would appear to send a message out to everyone in the group on my end but in reality the message would go out to only the first recipient (which can easily be chalked down to an incorrect API call, a failure to prep some sort of “group message envelope” or anything of that nature), sending a group message to n recipients would result in that message going out to n random contacts!
That, to me, was quite the shock. In attempting to reverse-engineer the iMessage protocol, I’d well accepted that I might trash my six years of iMessages I’d proudly maintained, that I might mess up my iMessage account, or that I would never be able to rely on my iMessage port to accurately tell me whether a message I’d sent out had actually gone through or not. But what I never in a million years expected was that I would instruct iMessage, through what appeared to be an official-yet-hidden API for sending such messages, to send out a message to named individuals and have iMessage respond by sending out the message to different recipients entirely.
I was able to bluff my way out of some of these mixups, but others were a little more awkward to explain. I’m a bit of a “contact info hoarder” and my address book is littered with hundreds of people whom I have not spoken to in years. Imagine their surprise when they received a message from me with no context, a message that was part of a separate conversation, one which they were not privy to altogether?
I’ll be honest. That almost worked. It almost stopped me from continuing to try getting group messaging to work. The thought of a message intended for a work colleague ending up reaching someone I hadn’t spoken to in years was scary. Not as scary as that one time I meant to complain about Charlie to Jane and, with Charlie on my mind and seething with anger, I texted Charlie instead… but, hey, that could also happen. But something about just how unfair this particular deterrent seemed further hardened my resolve, and I decided I’d get this working no matter the cost.
I did, however, decide to play things safe and created a new user account with no pre-existing contacts to serve as my testbed to safeguard against future booby traps; something I probably should have done from the start, if I’m being honest with myself. And I decided to try a different approach, reasoning that if I knew Apple had purposely sabotaged iMessage via AppleScript in an OS update, perhaps they had only added some of these protections to OS X in a subsequent update, too. Perhaps they had slipped up and let out a release that didn’t have these layers of protection and various traps lying in wait for innocent, curious individuals to fall prey to.
Deploying an old version of OS X to serve as my actual, final solution was never on the agenda. After all, I was aware that old versions of OS X did not sufficiently lock down AppleScript in such a way that would prevent the use of OS X as an iMessage proxy, and could have resorted to that approach from the very start. But I wanted to do this and I wanted to do it right. I wanted an elegant approach that I could deploy on the same machine I still used from time to time, without being stuck on an ancient (and insecure) legacy version of OS X that could suddenly be blocked from the iMessage network at any point. I wanted to piggyback on an API that would remain updated and maintained by Apple, that would keep working even as macOS and the iMessage protocol expanded and evolved in the years to come. I was willing to put in the time and effort to do it correctly now so that I wouldn’t have to invest more time on this in the future, time that might be better spent on other, more productive tasks (one can dream, right?).
Going back to an old seed of OS X to discover differences in the messaging APIs that I could use to arrive at a solution that worked for modern macOS was definitely not out of the question, though. The only catch was that Apple had largely succeeded in blocking virtual machines from using iMessage, even if all other features of OS X/macOS would work. It would be inconvenient, but thanks to the time-tested capabilities of Time Machine, I was able to make a backup of my never-formatted, always updated or migrated macOS installation and then install various older versions of OS X I’d found in an effort to track down changes that might give me a clue that would finally let me lay the final piece of the puzzle in place and complete this challenge I’d brought on myself.
Alas, as much as I’d love to tell you it weren’t the case, we now arrive at the sad part of the story. I did figure it out, and I do now have a fully working iMessage client that runs on Windows and sends messages via an up-to-date macOS High Sierra installation running on my rMBP. Messages send just fine to individuals and groups, both on and off the iMessage network. Incoming messages and outgoing messages both fly in and out just swimmingly. Attachments are sent and received without a hitch. Group conversations can not only be sent and received – to their intended recipients, no less! – but even instantiated from my Windows PCs. And everything syncs back to my MacBook and my iPhone, even marking conversations as read on iOS when they’re opened from Windows.
But what I cannot do is share my final solution with the world, as much as it pains me not to. Information yearns to be free and knowledge deserves to be shared. This knowledge most of all, given the efforts I’ve gone through to secure it, against all the odds and challenges. I love open source, and I’ve made sure that at NeoSmart Technologies we continuously open source libraries and solutions to technical challenges, so that we may give back to the open source development community that has made so much progress and innovation possible by allowing us all to build on the shoulders of giants. I take it upon myself to donate my time and money to contribute back to both open source and closed-but-free software that I found useful. I would love nothing more than to share with the world what I’ve built, if only to hear feedback that would make it better.
In the pursuit of perfection, I unwittingly built the perfect trap for myself. Apple has finally bested me here. What they weren’t able to do in code, they have done with policy: I love my iMessage too much, and I am afraid they’ll take it away from me if I do. I have spent countless hours perfecting this solution, and I have too much invested in iMessage for Windows, so lovingly polished to a near-perfect fit and finish, to throw it all away.
It is the ultimate irony: all the time and effort I spent fighting the dragons Apple left behind to secure the treasure I sought to free has made me cherish it all the more, and left me incapable of breaking it free. The thought that in the blink of an eye, in a fit of revenge I know would be fast in coming, Apple could take away from me all that I’ve fought so long and hard to obtain is an impossibly bitter pill to swallow. I am now perhaps as attached to iMessage as the developers at Apple that first brought it to life were, if not more so.
I have, however, left hints here for those that wish to embark on a similar journey. Perhaps in fighting a few of the same battles the overwhelming need I now feel to protect that which has been so painstakingly created will be instilled in those that gleam insight from my experience, and they too will protect it; letting iMessage spark to life in a few carefully kept pockets of safety outside the ominous, towering walls of Apple’s beautiful garden.
Some public good did come out of all this, however: no less than half-a-dozen libraries originally necessitated by this project and later fleshed out and documented were written and open sourced to the community (on GitHub under the MIT license). More in the captions accompanying the feature screenshots below.
WSL is the Windows Subsystem for Linux, also known as “Bash on Ubuntu on Windows,” offers a native Linux-in-a-shell with nice (and growing) integration with Windows. Check it out! ↩
For example, hackintosh PCs that act identical to genuine Mac hardware can use all macOS features except iMessage, which has been locked – so far as we can tell – to only allow network activation on devices that have “blessed” NICs that officially shipped with Apple hardware. ↩
The WhatsApp of today bears little resemblance to that of yesteryear, when security was merely an afterthought and the API was openly reverse-engineered and fairly well documented. ↩
Apple has never officially recognized the breakage of the AppleScript API, which blocked many operations pertaining to accessing existing chats and creating new ones, in subsequent macOS releases. ↩
i.e. never previously messaged from Messages.app on macOS or the iPhone ↩
As Slack does not allow whitespace or special symbols in group names or duplicate group names, using actual user names was out of the question. Using phone numbers or email addresses with the
@symbol swapped was a brittle but workable alternative. I’m sure better options existed. ↩
Ubuntu and Slack, why I’ll never know. Yes, I know I can change it. No, that’s not the point. ↩
No more “Joe the Plumber,” once you know his last name, after all! ↩
Messages.app doesn’t even offer AppleScript integration for incoming SMS messages, meaning there was no AS-based approach to getting notifications on incoming text messages from users not on the iMessage network. ↩