What's New in the First Public CTP of SQL Server 2019

By: Aaron Bertrand || Related Tips:More >SQL Server 2019

Problem

The first public CTP of SQL Server 2019 was released today and, let me tell you, it is overflowing with enhancements and new features (many of which you will also find, in preview form, in Azure SQL Database). I've had the luxury of early access, allowing me to share my experience with many of the changes, even if only at a high level. You can also check out the latest posts from the SQL Server team and the updated documentation .

Solution

I'm going to discuss several of the new engine features, at a high level, under five categories: performance, troubleshooting, security, availability, and development. At this time, I have more details for some features than others, and in fact have already written full posts detailing a few. I will come back and update these sections as more documentation and articles are made available. Rest assured, this is not an exhaustive look, only the things I've checked out, up to and including CTP 2.0. There is plenty more to come.

Performance Table variable deferred compilation

Table variables have a bit of a bad reputation, mostly because of estimates. By default, SQL Server will estimate that only one row will ever be present in the table variable, which can lead to some interesting plan selection when the row counts end up being much higher. A typical workaround to avoid this is to use OPTION (RECOMPILE) , but this requires a code change, and it is wasteful to recompile every single time when row counts are usually the same. Trace flag 2453 was introduced to simulate the recompile behavior, but requires running under a trace flag, and only happens when a significant change in rows is observed.

In 150 compatibility, you now get deferred compilation when table variables are involved, meaning it won't build a plan until the table variable has been populated once. Estimates will be based on the first use of that table variable, and no recompiles will occur after that. It is a compromise between recompiling all the time to get accurate estimates every time, and recompiling never and always having an estimate of 1. This is great if your row counts remain relatively stable (and better if that number is further away from 1), but may be less beneficial if they fluctuate widely.

I dug deeper into this functionality in a recent tip, Table Variable Deferred Compilation in SQL Server , and Brent Ozar has talked about it too, in Faster Table Variables (And New Parameter Sniffing Issues) .

Row mode memory grant feedback

SQL Server 2017 introduced batch mode memory grant feedback, which is described in detail here . Essentially, for any memory grant involved with a plan involving batch mode operators, SQL Server will evaluate the memory used by the query, and compare it to the memory requested. If the memory requested is too low or too high, leading to spills or wasted memory, it will adjust the memory grant associated with the execution plan the next time it runs. This will either reduce the grant to allow higher concurrency, or increase it to improve performance.

Now we get this behavior for row mode queries as well, under compatibility level 150. If a query is found to spill to disk, the memory grant will be increased for subsequent executions. If the actual memory used by the query is less than half the memory granted, subsequent grant requests will be lower. Brent Ozar goes into more detail in his post on Adaptive Memory Grants .

Batch mode over rowstore

Since SQL Server 2012, queries against tables with columnstore indexes have benefitted from the performance enhancements of batch mode. The improvements transpire due to the query processor performing batch processing rather than row-by-row. The rows also surface from the storage engine in batches, and parallelism exchange operators can be avoided. Paul White ( @SQL_Kiwi ) reminded me that if you introduce an empty columnstore table to make batch mode operations possible, the surfaced rows are gathered into batches by an invisible operator. However, this hack may negate any improvements gained from the batch mode processing. Some info in this Stack Exchange answer .

Under compatibility level 150, SQL Server 2019 will automatically choose batch mode in certain “sweet spot” cases, even when no columnstore index is present. You might think, well, why not just create a columnstore index and be done with it? Or keep using the hack mentioned above? This is being extended to traditional rowstore objects because columnstore indexes are not always possible, for a variety of reasons, including lack of feature support (e.g. triggers), the overhead in an update- or delete-heavy workload, or lack of 3 rd party vendor support. And the hack is simply bad news.

I created a very simple table with 10 million rows and a single clustered index on an integer column, then ran this query:

SELECT sa5, sa2, SUM(i1), SUM(i2), COUNT(*)
FROM dbo.FactTable
WHERE i1 > 100000
GROUP BY sa5, sa2
ORDER BY sa5, sa2;

The plan clearly shows a clustered index seek and parallelism, but no sign of a columnstore index anywhere (as shown in SentryOne Plan Explorer , a free query tuning tool):

And if you dig a little deeper, you can see that almost all of the operators ran in batch mode, even the sort and the compute scalar:

You can disable this functionality by staying in a lower compatibility level, by changing a database scoped configuration (to arrive in a future CTP), or by using the DISALLOW_BATCH_MODE query hint:

SELECT … OPTION (USE HINT ('DISALLOW_BATCH_MODE'));

In this case the above query plan has an additional exchange operator, all the operators execute in row mode, and the query takes almost three times as long to run:

You can see it to some degree in the diagram, but in the plan tree details, you can also see the impact of the predicate not truly being able to eliminate rows until after the sort: