Introduction to Database Proxies
Database Proxies can enhance performance and security in complex, high-traffic distributed systems built with Microservices.
Imagine a complex distributed system that relies heavily on data, where each microservice or a team connects to a database individually (it can be a shared database or specific/isolated ones). Such a complex platform requires things like centralized monitoring, query validations, alerting, custom sharding, and better security, you name it. While you can get a lot of that from your database server, implementing a database proxy may be a better approach (if you’re ready to invest in it).
The main advantage of using a database proxy is that it isolates the database topology from the application layer, so that developers don’t need to know about clusters, nodes and internals of the Data layer (to some extent obviously).
Database Proxy Use Cases
Let’s dive into the various ways database proxies can empower your development teams, enhance security, and optimize database performance.
Intercept SQL queries from applications and dynamically route them to the correct database/table (for example custom sharding). Figma is doing exactly that with their internal Postgres proxy.
Parse/Analyze/Validate SQL queries from developers and enrich the response with additional information. Could be useful to tell applications which tables are going to be deprecated.
Scalability and schema changes without impacting applications. Platform/Database teams can independently change the schema without rewriting hundreds of microservices. The ability to transparently add or remove nodes in the database cluster without having to reconfigure or restart an application.
Enforce security policies and perform authentication and authorization checks to ensure that only authorized clients can access the database. Direct access to a database can be also forbidden.
Improve the performance of database communication by centrally managing the pool of connections, utilizing cache technics, etc.
Centralized observability. Get notified when an application is using a deprecated table, and much more.
When to use a Database Proxy
Not all systems need a database proxy, especially early on. Here is the general guideline for when it might become necessary:
You have multiple development teams split by different disciplines: for example multiple Backend Teams, Data Engineering team.
You have a Platform/Database team to be an owner of that. Though other teams can own it as well.
Your system is distributed, and you maintain many microservices and many databases.
Your system is data-heavy.
You need better Security and Observability.
