Quick! Name a technology sector that has nearly 400 options competing for your attention, generated over $80 billion in revenue last year while also accelerating its growth rate, continues to give rise to startups decades after its inception, and generated the most job postings of any programming language in 2014. Your estimate of “database” was correct.
Why is this decades-old market currently so wildly popular? Despite slow growth for database leaders like Oracle, the market is rising.
As I’ve stated, one major factor is the cloud, but the main factor is that data is becoming increasingly important to every company and that heterogeneous, unstructured data results in the creation of new databases that need to be managed.
Evolution and revolution are met:
This isn’t how markets are intended to operate. Product categories tend to grow initially before declining over time and being replaced by other things.
For instance, Microsoft made billions in the operating system (OS) business, but OSs aren’t something we think about all that often these days.
On the desktop, tools like ChromeOS have made it very clear that the browser and the web are what matter most, and corporations are increasingly considering serverless architectures for servers.
Or perhaps you can still recall how hot the markets were for app servers, enterprise resource planning (ERP), and enterprise content management (ECM)?
These items, or some variations of them, are still used by businesses, although they are not seen as growth markets.
Arguably, databases ought to be uniform. Early in the 1970s, relational databases were created, and companies like Oracle, Microsoft, and IBM quickly established substantial businesses to market and support them. This market should be nearing its end right now, but we’re not.
Although these suppliers’ database revenue growth has slowed, the market as a whole has not. Although more of their clients are experimenting with PostgreSQL, cloud databases are now being used by even more.
AWS and other cloud powerhouses are even attempting to compete against both by providing managed PostgreSQL services.
Additionally, there has been a significant and ongoing increase in so-called “NoSQL” databases. The reason I don’t particularly like the trend is that databases like MongoDB, Apache Cassandra, Neo4j, DynamoDB, Redis, and others are being appreciated for what they are, namely flexible, capable of handling the surge in unstructured data, and horizontally scalable.
In fact, throughout the last nine years, including the last year, relational databases have decreased in comparison to non-relational databases, with the notable exception of PostgreSQL, according to DB-Engines (as illustrated here).
That’s not to say that the use of SQL and relational databases is declining. In fact, as indicated by job posts, SQL adoption is advancing.
Businesses are becoming more interested in developers who can use easy-to-use SQL to query the databases that have been powering their businesses for years. Because it has been a fantastic workhorse for the business, SQL is well-liked.
Enterprises are also clearly searching for developers who can assist them in querying new data sources and types, which frequently won’t require SQL, at the same time.
In other words, it’s not an either/or choice. It’s a “and” situation for businesses of any respectable scale. Businesses are merely attempting to maximize the value of their data and use the finest database for the task at hand.
Reorganizing the market: Zilliz, the organization behind the free and open-source vector database Milvus, recently raised $60 million in addition to the $43 million it had already raised.
Unfamiliar with vector databases? Not by yourself. In order to manage vector embeddings, a vector database is used. According to Frank Liu of Zilliz:
Machine learning models that have been trained to comprehend [unstructured] data are being used more frequently as a result of the increasing prevalence of unstructured data.
A well-known early example of this is the natural language processing (NLP) method word2vec, which employs a neural network to learn word connections.
The word2vec model may convert a list of floating point values, or vectors, from a single word (in a variety of languages, not only English).
The phrase “embedding vectors” refers to the fact that due to the way models are trained, vectors that are close to one another reflect words that are similar to one another.
In order to grasp the content rather than the keywords connected with it, vector databases are ideal for tasks like picture search or searching within the video, audio, or other types of unstructured data.
I’m not trying to provide a vector database tutorial here. Instead, it demonstrates how the database business will continue to expand along with the ongoing expansion of structured and, in particular, unstructured data. At the same time, new database-related strategies will emerge.
