Continuation from the previous 90 parts, the whole series can be found at http://www.nikoport.com/columnstore/ .
Given the improvements and the availability of the of the programability surface for every edition (with some insignificant & logical limitations) that I have blogged about in
SQL Server 2016 SP1 Programmability Surface for everyone! , I believe everyone using Microsoft Data Platform has rejoyced greatly. Of course, now everyone can have Columnstore Indexes on every SQL Server edition!
There are some noticeable limitations that were announced right from the start, such as the maximum size of the Columnstore Object Pool (you can find more information about it here Columnstore Indexes part 38 (“Memory Structures”) ), but there are more limitations to the Standard Editions and inferior ones and it is extremely important to know them, to understand them in order to make the right decision when your Business is ready/needed to upgrade to the Enterprise Edition of the SQL Server.
I have run a number of possible tests against freshly restored Standard Edition of SQL Server 2016 with Service Pack 1 and these are a number of items that I have found to be functioning differently in comparison to the Enterprise Edition of SQL Server 2016 Service Pack1, and this blog post will go through them.
First of all there is a limitation on the Columnstore Object Pool size with 32 GB , which should not be the biggest problem in the real world, because you can still run your queries against TB-big tables, you will simply not being able to keep them inside your memory all the time.
For the rest of the discoveries, we shall better run some queries and for that I have set up an 8-core Virtual Machine in Azure with a Standard Edition of SQL Server 2016 Service Pack 1:
For the setup I will use my favourite ContosoRetailDW free database and the traditional script to restore the database, drop the Primary Clustered Key from the FactOnlineSales table and to create a Clustered Columnstore Index on it:
USE [master] alterdatabaseContosoRetailDW setSINGLE_USERWITHROLLBACKIMMEDIATE; RESTOREDATABASE [ContosoRetailDW] FROMDISK = N'C:\Install\ContosoRetailDW.bak' WITHFILE = 1, MOVE N'ContosoRetailDW2.0' TO N'C:\Data\ContosoRetailDW.mdf', MOVE N'ContosoRetailDW2.0_log' TO N'C:\Data\ContosoRetailDW.ldf', NOUNLOAD,STATS = 5; alterdatabaseContosoRetailDW setMULTI_USER; GO ALTERDATABASE [ContosoRetailDW] SETCOMPATIBILITY_LEVEL = 130 GO ALTERDATABASE [ContosoRetailDW] MODIFYFILE ( NAME = N'ContosoRetailDW2.0', SIZE = 2000000KB , FILEGROWTH = 128000KB ) GO ALTERDATABASE [ContosoRetailDW] MODIFYFILE ( NAME = N'ContosoRetailDW2.0_log', SIZE = 400000KB , FILEGROWTH = 256000KB ) GO use ContosoRetailDW; -- DropthePrimaryKeyfromtheFactOnlineSalestable ALTERTABLEdbo.[FactOnlineSales] DROPCONSTRAINT [PK_FactOnlineSales_SalesKey]; -- CreateClusteredColumnstreoIndexontheFactOnlineSalestable createclusteredcolumnstoreIndexPK_FactOnlineSales ondbo.FactOnlineSales;Now, let’s run a couple of queries against our Columnstore Table to see what is functioning differently, when compared to the Enterprise Edition, but before that to make sure I obtain the parallelism easily without torturing Query Optimiser and making the understanding of the article more easy I have configured a couple of settings for my SQL Server Instance ( PLEASE DO NOT DO THIS ON THE PRODUCTION INSTANCE! )
Besides the Max Threshold for Parallelism set to 1, Maximum Degree Of Parallelism set to 8, I have set a whole set of the options through DSC (Desired State Configuration) that we typically apply on the windows Servers that running SQL Server (including max server memory, IFI, etc).
Let’s advance and execute a simple query that will be forced to run in parallel, to check if it is running with Batch Execution Mode and if everything corresponds to the typical processing of the Enterprise Edition:
setstatisticstime, ioon selectsum(SalesQuantity * UnitPrice ) fromdbo.FactOnlineSalesIf we look at the execution plan of our query, it looks quite expectable running in paralleled doing all the task one would expect it to do:
But should we take a look at the properties of the Columnstore Index scan, there are a couple of very interesting details to uncover:
First of all the Columnstore Index Scan runs with Batch Execution Mode, and that should make everyone very happy (there has never been a Batch Mode on the Standard Edition of SQL Server before 2016 with Service Pack 1), but take a more detailed look at the Actual Number of Batches it shows that there are only 2 cores doing the work instead of 8 that I have configured for the SQL Server instance and I would expect to function in the Enterprise Edition of the SQL Server.
Taking a more detailed look at the properties of the query, shows that indeed the used degree of the parallelism is capped to 2, even though when looking into the XML of the execution plan reveals EstimatedAvailableDegreeOfParallelism = “4”.
This is definitely a cap of the Standard Edition, that will prevent your Columnstore Queries that run with Batch Execution Mode to use more than 2 Cores in Parallel. Should your queries run in the Row Execution Mode, all of the configured CPU Cores will be potentially used for processing.
Also notice, I can easily run 4 parallel queries for my SQL Server Instance thus using the potential of my SQL Server Instance to the maximum. Notice that for the Web & Express Edition, the limitation will be stronger, making queries to run in Batch Mode in Single-Threaded fashion only.
This is not the totally unexpected limitation of the Standard Edition, but one should be aware that it exists.
Let’s run another query, trying to see if the aggregate pushdown works correctly (for more information on Aggregate Pushdown, please consult the following article Columnstore Indexes part 59 (“Aggregate Pushdown”) ):
setstatisticstime, ioon selectmax(SalesQuantity) fromdbo.FactOnlineSales;The execution time on my VM was very fast, making the total elapsed time to be just 63 ms, but if you look at the execution plan, you will get a little bit surprised, since it shows no aggregate pushdown taking place:
In Enterprise Edition the functioning Aggregate Pushdown looks in the following way in the execution plan, where no rows are coming of the Columnstore Index Scan:
This means that Aggregate Pushdown is disabled for the Non-Enterprise Editions of SQL Server and that’s a pity, but still your workloads on Standard Edition will function extremel