Posts Tagged ‘Column store Index’

In my previous post we had discussed about column store index (non clustered), after 2 years in Microsoft’s new version SQL 2014 they have introduced the clustered column store index. The main limitation of the column store indexed tables were ready only (it was not apt for OLTP database), in the latest version Microsoft has introduced clustered column store index with eliminating data manipulation restriction.

Let us discuss how Microsoft achieved this feature.

Delta storage (Insert):  The data insert operations are costly in case of column store index, to over come the scenario SQL has introduced this new storage space called Delta storage. In delta store data would be stored as a traditional method (row store method) and data would be moved column store method using a mechanism called tuple mover. Tuple mover is mechanism that runs like a job in every 5 minutes and move all the rows to column store index without taking the table offline. Delta store is not used  in case of bulk insert.

The tuple mover waits delta storage to reach a limit of 1048576 rows, if the delta storage reached this number of rows the particular delta store would be closed and that would be ready to move column store index. If tuple mover finds a delta storage like this, tuple mover pull data from the storage and compress the data and then move to column store index. Mean while data can be inserted to another available delta storage so this process can be performed online.

Delete Bitmap(Delete) : To perform a efficient delete from column store index is a hard task, to over come SQL has introduced a segment called Delete bitmap this contains the information of deleted rows. When user requests for rows SQL retrieve records which are not exist in Delete bit map. So basically there is no changes happened in column store index when delete operations perform and the deleted data exist in the index pages. Index pages are get refreshed only when we rebuild or tuple mover.

Update : We have already discussed about how delete and insert works in column store indexes,for update SQL uses both delete and insert operations. SQL delete existing row using delete bit map method (virtual delete) and insert rows to the delta stores as usual insert operation. When next time select request for the row, delete bit map hides the old row and show the new row from Delta store.

ColumnStore2014_01

Now we have understood how the insert delete operations are taking place in column store index, in SQL 2014 we can create only one column store index on a table.

Non clustered Column store indexes are introduced in SQL 2012, here we are going to discus about structure , limitations and advantages of non clustered column store index based on SQL 2012.

An index is nothing but arrangement of data to reach each rows in easiest way(here are we are not discussing about selectivity and density since the topic is overview on column store index). When column store indexes came into picture once People started thinking about column instead of rows.Either user interested in a single column or all the columns SQL has to load entire row in to memory and show only the requested columns to the user. If the user is only interested in a single column why should system waste time to read entire row.

Let use see how SQL deals with it and how data being stored in case of column store index.

Row store method data storage

column-store-image001

Column store method data storage

column-store-image002

Based on the above pictures you can see in case of column store same column data is stored in single page, this improves the compression (since data is homogeneous) and If you are interested on a single column then SQL search engine has to read only one column, the over head caused by row store method has been eliminated by column store index.

Limitations

  1. The table will become read only if you create a column store index.
  2.  Sparse columns are not supported.
  3. Cannot be created on a view or indexed view.
  4. Cannot be created by using the INCLUDE keyword.
  5. Cannot include the ASC or DESC keywords for sorting the index. Columnstore indexes are ordered according to the compression algorithms. Sorting would eliminate many of the performance benefits.

Note :  if you are interested to pull all the rows from a table definitely column store index is not going to help you in terms of performance.

column-store-image003

The above screenshot is from SQL 2012 instance running in my PC.  I have created a table and created a non clustered column store index on it. I tried to insert a row after that then I got that error.

Now I suggest you to create one more column store index on the same and see what happens , please do comment the result below.

Note : The post is described column store index based on SQL2012 version, few of the limitations has eliminated in the latest version.

Ref : MSDN more details about column store index.