March 30, 2024 - How I replaced my vendor with a very small C# script | Mistaking an Opportunity as a Problem

I may have accidentally fired a vendor, drove a small business to close shop, and maybe missed a half million dollar market opportunity by making a free app.

Early in my career, I worked for a B2B SaaS company whose marketing function was doing the latest lead-gen tactic of the day: buying nearly-cold email lists from vendors who claimed that the people we'd end up emailing had some behavioral affinity for the space we were in. Sounds icky right? Except today, B2B software sales people just buy the email lists from vendors or scrape it off Linkedin in and don't even bother with behavioral affinity matching most of the time. Same thing, its just easier and cheaper to have sales people do it themselves.

Part of my job, as an "analyst" within the marketing function, was to ensure the that my company only paid for the portions of a vendor's email list that we didn't already have in our marketing database. Of course, we didn't want to give our vendor access to all of our records to scrub their email list against ours, and neither did they want to give us the whole list for us to scrub ourselves. A common solution at the time was to use third party vendors for whom we could upload our email list to, the vendor theirs, and the third party vendor would return the matched email addresses to the list provider and/or the non-matched email addresses to us. My company had a paid recurring subscription to one such vendor.

I liked the list scrubbing vendor and their product just fine. But for reasons I don't remember, some of the vendors selling us email lists couldn't or wouldn't use the third party vendor to scrub their lists. It was my job to figure out a solution to get the email list that my company had already paid for (days ago!) scrubbed and in to our email database - ASAP!

In the process of learning about how our list scrubbing vendor worked, I must have read up on what a hashing algorithm is, how it works, and how it is different than encryption. I actually don't remember how or why I learned this, but I found my way in to the C# programming language, which I was already somewhat familiar with, to see if I could create my own list scrubbing tool. Back then, MD5 was the hashing algorithm used for such a process.

For the non-technical, a hashing algorithm is (in gross over-generalized and hand wavy terms) sort of like one-way encryption. It might give us the fingerprint of the underlying data without revealing the identity of the fingerprint's owner. Hashing something looks a lot like encryption though: If your password was 1-2-3-4-5 and you put it through an MD5 hashing algorithm, it's output would look like 601bcbd12379a3a03bcbad773954109d. Something hashed isn't intended to be decrypted. Just sorted and compared. In fact, it was initially thought that hashes can't be decrypted (though we would later find out that to be not entirely accurate). If you trust it, ask your preferred Stochastic Parrot for an ELI5 of Hashing Algorithms.

Fortunately for me, the C# programming language already had MD5 support in an existing library. So I created a script, and then a desktop app to hash and compare email lists. It was a very straight forward endeavor. After I tested a variety of CSV lists with the app, I live-tested it with an email list vendor over the phone. I emailed them the zipped binary, told them how to unzip and run it, and instructed them on how to use it to compare my MD5 hashed list with their own list. Aside from a complaint that the app's progress counter stopped incrementing through their list at 65,535 rows, my vendor assured me the whole list was scrubbed and its output saved to the appropriate CSV - which he then sent me.

I just gave away, free of license or any financial request, a desktop app that could be infinitely copied and shared, for which others were charging a subscription fee of at least a couple thousand US dollars per month. I wasn't trying to undercut our list scrubbing vendor or undercut any of the small businesses in the same space. I was actually just too focused on solving the problem to think about the opportunity.

The market capitalization and lifetime business opportunity for such a product couldn't ever have been that big. I'm generously guessing a million US dollars or less. I vaguely even recall discussing the market opportunity with a couple coworkers and being told by our boss it was too small for my employer to worry about. That was an implied green-light to do what I want outside of work, but I considered the problem solved, and I was too focused on starting my upcoming MBA program 🙃.