FinOps, or Financial Operations, is defined as “a framework and cultural practice that focuses on bringing financial accountability to cloud spending in a way that balances cost, operational efficiency, and quality.” Through FinOps, cross-functional teams such as finance, planning, product, and operations work together to manage cloud costs effectively and maximize business value. Additionally, a sustainability perspective is inherent in the concept as a byproduct. Note: FinOps can also be implemented in on-premises environments during data migrations.
Data migrations are typically large-scale projects requiring cross-functional expertise and have a significant impact on business. Data migration is the process of selecting, preparing, extracting, and possibly transforming data and permanently moving it from one data environment to another. SQL Server platform data migrations may occur for various reasons, such as server or storage replacements, consolidation, and data center relocation from on-premises to a public cloud like Microsoft Azure. Key drivers for moving to Azure include cost factors and their transparency, the need for databases to be close to applications (cloud-based applications), security, and operational efficiency. However, the costs of data platforms are not always automatically cheaper in the public cloud than on-premises, especially if the FinOps component is overlooked.
SQL Server data migrations are typically performed as automated as possible, freeing personnel from repetitive routines and avoiding errors common in manual repetitive work. Although migrations are often highly automated, they must be monitored, and migration successes must always be carefully verified: “What goes out, must come in.”
Components of FinOps
FinOps can be divided into five different components:
- Visibility and Accountability
- Cost Optimization
- Operational Efficiency
- Effective Teams
- Data-Driven Decision-Making
Visibility and accountability refer to the real-time transparency, predictability, and control of costs among different stakeholders.
The goal of cost optimization is to achieve savings and a better Total Cost of Ownership (TCO) without compromising the performance or availability of services. This includes concepts such as right-sizing, consolidation, and need-based dynamic scaling, which are some of the advantages of a public cloud like Azure.
Operational efficiency means that cost monitoring is continuous and comprehensive, allowing quick reactions to changing needs.
Effective teams mean that data engineers, when designing and further developing the data platform, understand and control the impact of architecture and configurations on the data platform's costs. This, in turn, contributes to the distribution of responsibility for overall costs.
Data-driven decision-making in FinOps relies on reliable data, enabling organizations to better anticipate and manage their costs.
The Role of FinOps in Data Migrations
FinOps plays a significant role in data migrations, especially as organizations move workloads and data from local environments to the cloud. Considering the costs, complexity, and financial impact of cloud migrations, FinOps can provide a framework for effective cost management and optimization.
In data migrations, FinOps can be divided into the following areas:
- Pre-Migration Planning and Cost Estimation
- Cost Optimization During Migration
- Post-Migration Capacity and Cost Optimization
- Governance and Accountability
- Data-Driven Insights and Reporting
Pre-Migration Planning and Cost Estimation
Pre-migration planning and cost estimation are divided into the following sub-areas:
- SQL Server performance data monitoring
- Budgeting, forecasting, and capacity planning
- Setting goals (KPIs)
SQL Server performance data should be collected from all SQL Servers, instances, and databases so that capacity planning and consolidation can be carried out comprehensively at all levels of the data platform. Suitable products for this purpose include specialized tools for SQL Server monitoring and capacity planning, such as SQL Governor, which can also identify unused database resources at the early stages of the project, avoiding unnecessary migration to the new environment and additional costs.
Budgeting and forecasting are best achieved by combining empirical knowledge from previous similar migrations and estimating capacity requirements by calculating different scenarios for logical areas, such as development, testing, and production environments by business area.
Capacity planning can apply the following methods to server clusters, servers, instances, or related:
- Lift and Shift (moving workloads to the new environment “as is”)
- Right-sizing, i.e., resizing servers or Managed Instances
- Server-level consolidation (optimizing the total number of hosts)
- Instance-level consolidation (optimizing the number of hosted virtual machines, Managed Instance pools, or physical servers)
- Database-level consolidation (harmonizing workloads even at the instance level)
In goal-setting, budgets and financial KPIs are defined within which the entire data migration should be implemented, achieving the desired ROI.
Cost Optimization During Migration
FinOps teams monitor migration costs in real-time and identify potential unexpected costs or resource spikes to keep expenses under control. FinOps best practices and capacity planning software, such as SQL Governor, help adjust resource usage significantly based on historical forecasts and real-time needs, avoiding service overprovisioning. However, it may be that, for example, with SQL Server version upgrades, especially if the previous version is already outdated (2008R2 – 2016), the new service platform requires optimizing a new database instance, such as adjusting instance and database-level configurations like Max Degree of Parallelism and Cost Threshold for Parallelism settings. Other optimizations should be left until after the migration when sufficient telemetry and SQL Server usage statistics have been collected in the new environment. However, Query Store should be enabled immediately after migration.
In cloud platforms, it is also possible to use various savings plans and long-term commitments related to migrations, which can provide additional savings.
Post-Migration Capacity and Cost Optimization
After migration, the FinOps team ensures that costs are accurately allocated to different departments or projects so that each team understands its expenses. The FinOps team also helps identify potentially remaining unused resources, facilitating their removal to avoid unnecessary costs. The FinOps team continuously optimizes resources in the new cloud environment, ensuring that workloads remain cost-effective over time.
Typically, after the first complete month (business cycle) post-migration, it is critically important to benchmark the performance of the new database environment at the server, instance, and database levels compared to the old environment. For example, the SQL Governor product includes a benchmarking tool that allows easy identification of system bottlenecks and progress through KPIs and graphs. If optimization needs arise, the SQL platform can be optimized by enforcing better-performing execution plans from Query Store, or even enabling Legacy Cardinality Estimator for problematic databases. It is not uncommon to have to rewrite some T-SQL code or tweak indexing if the application vendor permits. These issues often gradually emerge in already migrated environments, so some of these actions, especially related to indexing and code optimization, can only be performed post-migration. SQL Governor offers a useful toolset for resolving many of these issues.
Governance and Accountability
During migration, the FinOps team can establish practices such as guidelines on acceptable expenses, resource usage, and data egress restrictions to ensure cost control. Additionally, by providing real-time visibility into migration costs, FinOps enables planning teams to make cost-aware decisions and be accountable for their spending.
Data-Driven Insights and Reporting
FinOps provides regular reports and dashboards that allow stakeholders to understand migration costs, track progress relative to budget, and ensure organizational financial efficiency. For example, in Azure, billing telemetry and configuration-based forecasts are available “out of the box.” When combined with our AI-based SQL Governor product's ability to calculate capacity requirement forecasts and consolidation possibilities for existing databases, instances, and virtual machines over different configurations in the long term, we get a highly effective combination for optimizing TCO.
Lift and Shift – Poison for TCO
A word of caution regarding "Lift and Shift"—it's not all smooth sailing. The larger the database environment, the more issues will inevitably arise with this approach: A significant portion of databases are either overprovisioned or underprovisioned. Overprovisioned databases may not perform adequately, leading to downtime and hidden costs. Underprovisioned databases, on the other hand, can be unnecessarily costly by reserving extra capacity, especially in Azure VMs and Managed Instances, and Azure SQL’s Hyperscale isn’t free either.
The idea of Lift and Shift is good in that it minimizes visible migration costs and time-to-solution, but it can cause considerable headaches, especially when moving to the public cloud, such as Microsoft Azure. Although moving to Azure with Lift and Shift might initially sound easy, it often isn’t in practice: When you have to change storage capacity service tiers, guess the size or type of virtual machine from one to another, or encounter a high-availability configuration that cannot be downgraded later, it becomes problematic. Such decisions should be based on forward-looking telemetry data to ensure a sustainable solution. Simply looking at historical data isn’t enough.
Additionally, there are downtime and service disruptions caused by re-optimizations or re-configurations, which consume both money and time while lowering service SLAs. Servers, instances, and databases can also be optimized post-migration, but this process takes as much time as it would before the migration, leaving the total cost of investment uncertain. After service launch, you certainly do not want any additional downtime in your business-critical systems, especially in healthcare, industrial, or other time-critical environments.
An Alternative to Lift and Shift in Requirement Specification
Sometimes Lift and Shift is the only solution due to policy, commercial constraints, or time pressures. Regardless, a thorough requirement specification for the data migration project should always be conducted before starting the transfer to a new environment. This includes reviewing the organization's needs, including a business impact analysis, target architecture, and capacity plan, as well as a cost savings calculation and continuously updated risk analysis. Additionally, a migration path for various subcomponents is considered, typically covering a range of preparatory performance optimizations, right-sizing, and workload consolidation at the server, instance, and database levels. This ensures a high success rate right from the start and avoids Lift and Shift issues.
Summary
FinOps builds a strong financial backbone for data migrations, keeping budgets under control, especially when moving to the public cloud like Azure. The reward is resource savings, including money, energy, and the environment. Sustainably, one piece at a time. Are you interested in SQL Governor, FinOps, and data migrations? Please feel free to contact us!
Jani K. Savolainen
CEO & Chairman,
SQL Governor
