Are Databases a Commodity?
This picture is of Tom Coffing teaching back in the 1990s. Tom taught thousands of classes to hundreds of customers worldwide for over 30 years.
You don’t kick someone’s dog, yell at their children, and for heaven’s sake, mention anything negative about their database.
I received backlash from Teradata and Snowflake employees and customers last week for suggesting that the free query tools they provide are poor reflections of their brilliant databases. I say this with only love. I was on the front lines when Teradata was born, taught over 1,000 classes to the largest customers, and wrote 50 books on Teradata alone. I wrote another 35 books on every other database and watched each grow from its first release to great acceptance. I spent more time with each database than my family at times. However, one thing became clear to me 20 years ago. Every database vendor thinks the world begins and ends with their system. Vendors would be brokenhearted if they knew their customer was cheating on them with another database vendor. “How could you?”
Teradata promised they could keep all of your enterprise data in one place, and an unlimited amount of users could query simultaneously. But data is too big, there are too many users, and performance can’t even begin to be sustained unless you rob Fort Knox.
It was exactly 18 years ago, almost to the day, when I saw the future. I took a deep breath and said, “The database will be a commodity, and customers will need hundreds of databases to hold their data.” And I realized that a database is nothing more than something that stores your data and allows people to query it to get an answer set.
It was the second greatest moment of my 25-year career in data because I knew it was not about the database but about putting the power in the hands of the people inside the company. The future would be a software tool that queries, migrates, and joins data across all databases. Once the user gets many answer sets, they need to slice and dice the results inside their PC as intelligently as if it were a database.
I was not naïve about the challenge because I knew how difficult it was to get two different vendor systems to talk to one another. Teradata and Microsoft hate Oracle, and IBM hates them all, and Snowflake wants to take over everyone’s data, and each is offended that MySQL or Postgres are even in the mix, and so on.
I also knew that converting table structures, data types, and building load utilities to move data is the most difficult challenge.
That brings me to the greatest moment in my 43-year career in data. I spent the last 18 years on one project: to build the only software where everyone can point-and-click and migrate to and from every database, join across all databases in a single query, and get answer sets and slice and dice them every way possible. I even built a Super Join Builder to write the SQL for every user, producing federated queries across 20 different systems.
I called the software Nexus, the intersection between all databases, and it works incredibly. It doesn’t matter if you access one database platform or hundreds. You can query, join, migrate across them all, and slice and dice the results seven ways to Sunday. I had help from the largest banks, airlines, insurance companies, PC manufacturers, financial advising companies, hospitality, and package delivery companies because they bought Nexus and helped it grow with ideas and suggestions.
Many call me now and say, “I just moved 10,000 tables with the click of a button.” I say, “Of course!” Was it easy to build? No, it was a damn nightmare because I had to test and fail, test and fail, test and fail until I got it right. Are databases a commodity? Yes and no. Each has the same purpose and uses many of the same techniques, but each brings something brilliant to the table so take advantage of them all. Unfortunately, the free query tools they give out are a horrible idea because Nexus is a technology that gets the most out of every database.
The torch has been passed, and the paradigm has shifted. It is not about the database but the tool that provides access to any data at any time, no matter where it resides. I always wanted to create my own database, but I decided instead to create the only tool to access them all. So you know we truly all can get along!
The video below shows Nexus migrating and federating data brilliantly across multiple systems.
Every child should be able to dream.