We as OptimaData hope that all clients and other relations have been able to conclude 2022 nicely. It is time for a new chapter. In the year behind us, we received many questions about database consulting and were able to give just as much advice and implement solutions. So what were the most important and common issues we discovered and what lessons can you take into the new year?
Cloud infrastructure
What we saw a lot of in 2022 was database environments in "the" cloud where the infrastructure was not properly aligned with the resource needs of the databases. Customers were experiencing disruptions, deadlocks, extreme latency and poor overall performance. After our investigation, we found in many cases that the database had little or no scalability because, for cost reasons, a cloud infrastructure was chosen that did not match. Sometimes we also see it the other way around, that customers were surprised afterwards by an unexpectedly high bill because they had not configured the cloud environment correctly and started scaling resources automatically. There is no standard solution for these situations. Sometimes it is sufficient to optimize and fine tune the database configuration so that it still performs more than adequately on minimal resources. In other cases, it is necessary to configure the infrastructure differently or even move it to another cloud provider.
Cloud providers
Sometimes databases live in the "wrong house. It is too simplistic to say that Azure has the best cloud infrastructure, or just AWS or Google. Which cloud environment your database will feel most at home in depends on your database environment, the type of database, your needs and requirements, and the load. Of course, it makes a big difference how many nodes, clusters and regions you have set up your environment with. There are now some nice and very well-performing smaller cloud providers, such as UpCloud, that can also be very interesting in terms of cost structure. A good benchmark can add value here.
Maintenance
A recurring theme is database maintenance. The customer either does not have the right expertise in-house or insufficient capacity. But often the damage has already been done and there are major problems that can ultimately be traced back to overdue maintenance. So-called DBaaS environments also need to be checked regularly to make sure everything is still running smoothly. The much-discussed HealthCheck indicates the necessary points in time.
Monitoring
Again last year we saw many environments that were not or barely monitored. An organization then sails 'blind' for a long time and only when clear problems arise - or even when there is a failure - does it call for our help.
By arranging good monitoring with an appropriate set of alerts, you can detect changes in the behavior of the database in time so that you can act immediately, before the monitoring data has disappeared due to the passage of time and the change has become "the new normal. In the picture, the suddenly arising spikes are still outside the alerting phase, so you don't even notice this in the use of the database itself. Until a new peak appears on top of this and then the virtual frying pan is over. Then recall what caused it. There are standard and easily configurable open source (read: free) monitoring setups available for many databases, such as PMM or Zabbix.
Data model
Again, we also often saw ill-conceived or bad queries. Based on the idea "a database is a database," queries are created with a semantic tweak here and there for an SQL dialect. These queries often cause huge delays and can even stop entire processes. It is quite easy to use a free open source database. But too often developers see this as a black box. With an illogical data model, the blackbox can even become a Pandora's box. A Quickscan brings light back into the darkness.
Standard configuration
Downloading, installing and deploying an open source database is very easy. Even spinning up a Galera cluster is something some organizations can do just fine on their own. But then... Every database, data platform and combination or setup has specific and custom configuration. It is often underestimated, but this really is a profession. Things like temp files, cache, working memory, type of backup, disk-usage, all factors of influence. An interplay, which we also described in our blog about the similarity between grand prix and databases. Our experts are top drivers when it comes to setting up the best configuration for your application.
Finally
Besides these frequently recurring issues, we have also been busy with our "regular" work, such as migrations, designs, benchmarking, quick scans, healthchecks and daily management. On SQL Server, MySQL, MariaDB, PostgreSQL, MongoDB and Sybase databases.
Need a sparring session?
Do you think that any of the above issues may also be an issue in your database environment? Are you doing well or could you do better? Feel free to contact us, we would love to help you.