Introduction
As software engineers we are asked to build system that solve a whole host of problems. Overtime, we encounter similar problem or sometimes even the same one’s that various companies face. With that being said, we sometimes approach solving the problem with a mental model that has been tried, true, and tested. Why reinvent the wheel if the wheel is the right solution. In this article, I want to try and make the case that sometimes we can solve problems a bit differently by expanding the tools we leverage within our mental model of solution. More specifically I want to show how NoSQL databases can be used to solve problems that traditionally have been tackled using relational database (RDMS).
Lets face it, many systems for several decades now have been built upon relational databases. RDMS’s are extremely mature, powerful, and enable us to create data driven systems that solve an enormous amount of business problems. RDMS’s are great! That being said when we encounter new problems its quite natural as a software engineer to leverage an RDMS.
Let’s imagine a typical scenario we face as developers. Some project is initiated within a company where a new system is going to be built. There is a design and discovery phase where the project really takes shape. Early in the design and discovery phase, engineers have already started to identify the right tools to solve the problem. It’s at this point in the design that we may be doing ourselves a disservice as engineers in the sense that we are limiting our solutions to what we know and feel comfortable with. In the next few sections I want to describe a problem that could very easily be solved using an RDMS but may actually be better served by using a NoSQL database and in particular Cosmos DB. In short we want to try an break the mental model of using RDMS as a default data store. Frank Zappa has a famous quote that says “Without deviation from the norm, progress is not possible”.
Business Problem
For a moment lets imagine we are building out an e-commerce solution for a B2B company that sells window coverings. A distinguishing characteristic is that the company does not actually manufacture the window coverings but rather sells products manufactured but a few dozen of their partners.
During the discovery and design phase, we identified some important qualities and characteristics of the product catalog
- Each product has its own unique set of attributes and lots of them. In addition, products share a core set of some core common attributes across all products.
- The set of attributes can change frequently. Examples interlude pricing, availability, product options, descriptions. The list goes on and on as to what can change.
- Manufactures each have their own unique way of representing a product and the relevant attributes
- Querying and displaying the product and all the relevant attributes is what helps customers understand what they are purchasing.
We could go through an exhaustive analysis phase and design a relational model in which all the products and attributes can fit into a well defined relational schema. It’s doable but difficult. Not to mention that every time a manufacture changes their product structure it could have severe impact on the product scheme. This is the point in the project that if we consider a NoSQL approach, we see some very significant advantages to the tried and true RDMS approach.
NoSQL Advantages.
- We can create documents that represent each of our products that is schema-less. This means as our products and attributes change then so can our document storage with great simplicity and ease.
- All of our products can still be easily queried. In particular, Cosmos DB supports querying documents using the very familiar SQL syntax of an RDMS through a SQL API.
- NoSQL and in particular Cosmos DB databases are positioned to scale to extremely large volumes of data with the ease of a push of a button.
- NoSQL and in particular Cosmos DB databases are fast…very fast with guaranteed SLAs.
- One could argue that the Object to Relational Impedance Mismatch is solved by leveraging NoSQL capabilities.
Specific Examples
Lets drill a bit more into the details of our product catalog to exhibit how storing and querying our document catalog in a NoSQL database like CosmosDB has advantages . Take this sample JSON representation of a product.
In the RDMS world, we would likely have a table for the product, brand, category, images, tags and colors so we can store the relevant data about our product. Lets suppose that in the future the manufacture adds a new attribute for options. In that case we would have to go to our RDMS DB schema, add the option table, and then update our code that ingests the product information to save the new information to the data repository. Similarly, we would likely have to update our data access logic to be aware of the options table when querying our database.
The first important advantage of NoSQL is the fact that changes like this pose no problem. Within the NoSQL world, our product catalog can be schema-less. Where this has a distinct advantage over an RDMS is that when the new options attribute appears, we don’t need to change our data storage in order to store this data. Likewise, this new data is immediately available to query. In the case of Azure Cosmos DB we can query this product using the SQL API using very similar SQL syntax.
An example of this using Cosmos DB as our storage is detailed below. This example is selecting any products where the brand = ‘Bali’. This is very similar to how we would select in an RDMS except for we don’t have all the joins. We are using the data in the schema-less documents and querying the relevant data.
Similarly, we can write our SQL to use nested attributes in the document structure and reshape our return to produce new structures. In other words shape the return of data to fit our object model. In the following example we are joining on a sub document and selecting out just the size and path of images where the size is small.
The main point of this is that our schema-less documents are still very much query able using the familiar SQL syntax.
This is just a small example of how using a NoSQL database like CosmosDB may be better suited to solve problems that have traditionally been solved using RDMS’s. One could argue that is solves it better. The point being that if we open our mental model of solutions to new tools like NoSQL we may find we can build better systems.
The NoSQL database and in particular Cosmos DB is a technology that is well suited to fit into the solutions for many common use cases. These include
- IoT and Telematics Systems
- Retail and Marketing Systems
- Gaming
- Social Applications
- Recommendation Engines
Summary
The technology space is constantly evolving. That being said keeping our eyes and ears open to new technologies can result in better solutions for our applications. The NoSQL databases and in particular Cosmos DB definitely has some advantages and should be something that you should consider. If you would like to find out more about Cosmos DB, I invite you read this Cosmos DB primer article. At the very least I think NoSQL databases should absolutely be a consideration for you solutions and you may find its a better solution.