The Pros and Cons of Using T-SQL Soft Transactions

By: Tim Smith || Related Tips:More >Database Design

Problem

We were recently introduced to a data storage approach where we never remove or update data, but flag data that is no longer true or active along with sometimes replacing a value with a new value (a technique that is sometimes referred to as soft deletes and updates). While this may help some of our applications, since we have a tendency to reverse a data change and the time to restore is costly or causes other data dependencies to fail, what are some considerations of using this approach?

Solution

This can be a useful solution where we approach storing data by disabling the delete and update operation on the back-end. For a delete, we treat the record with an update to a flag column as false (such as IsActive, IsExists, or IsCurrent another derivative). For an update, we add a new record with our flag column as true, while the old record is labeled as false. For explanation purposes, we'll first look at what soft transactions are with an example where we perform them on a new table derived from an existing table and look at situations where we may want to consider them or avoid them completely.

What Is A Soft Transaction?

In our first step, we will take a table with a primary key and create a copy of this table with either all of the columns or a selection of columns. I suggest testing this by copying an existing table to test how soft transactions will function rather than adding a column to an existing table and assuming that this will function.

In the below example, we create a full copy of an existing table by using the SELECT * INTO approach, which will copy a full table into a new table. From there, we generate the create table script to verify that no primary key exists on the new table (we could also check through the interface as shown in the below image). After that's verified, we add a new column with a default bit value of 1 to the table called true, which reflects whether the current record in the database is the true value.

Why do we want the default value to be 1 as opposed to 0? Because every new value needs to be true when we perform a soft transaction - we're never removing records: in an update, we're adding a new record (true) and updating the old record to false and in a delete transaction we're updating an existing record to false. In addition, we do not allow for NULL values on this column, as a row is either true or false.

In review, our steps were:

Create a new table from an existing table, where the existing table has a primary key using the SELECT * INTO syntax (for testing to validate our design will function correctly) Verify the new table has no primary key Add a new bit column called True (or another derivative) with a default value of 1 and make sure it's not NULL. /*
-- Some data for testing in this example
CREATE TABLE [dbo].[ValveImport](
[ValveOutput] [int] NULL,
[ValveInput] [int] NULL,
[MeasurementTime] [datetime] NULL,
[ImportDate] [datetime] DEFAULT GETDATE(),
[ValveId] [varchar](12) PRIMARY KEY,
[ImportFlag] [int] NOT NULL
) ON [PRIMARY]
GO
INSERT INTO ValveImport (ValveOutput,ValveInput,MeasurementTime,ValveId,ImportFlag)
VALUES (91,100,'2017-01-01 01:35:28.277','00a6f80450',0,1)
, (91,100,'2017-01-01 01:35:28.277','02037071a7',0,1)
, (91,100,'2017-01-01 01:35:28.277','02b33372ec',0,1)
, (91,100,'2017-01-01 01:35:28.277','04e42273c2',0,1)
*/
---- Create new table with primary key from existing table:
SELECT *
INTO newValveImport
FROM ValveImport
/*
---- (Do not run - the table is created from above). Notice the new table (scripted) has no primary key :
CREATE TABLE [dbo].[newValveImport](
[ValveOutput] [int] NULL,
[ValveInput] [int] NULL,
[MeasurementTime] [datetime] NULL,
[ImportDate] [datetime] DEFAULT GETDATE(), ---- updated to match original table
[ValveId] [varchar](12) NOT NULL,
[ImportFlag] [int] NOT NULL
) ON [PRIMARY]
GO
*/
---- Add new column
ALTER TABLE newValveImport ADD True BIT DEFAULT (1) NOT NULL
SELECT *
FROM newValveImport

Now that we have our soft transaction table, we'll demonstrate how we would run transactions against it. The nature of soft transaction eliminates the DELETE operation - we will no longer run a DELETE command, as a soft delete only updates a record to True equaling 0 (false). A soft UPDATE adds a new record (true) and changes the previous existing record to false. Therefore, we've reduced our write operations to INSERTs and UPDATES, with the latter being more nuanced than a straight update.

When we add a new record with a new primary key value (ValveId in this example), we can simply run an INSERT statement, as our table will have no ValveIds with that new value. If a ValveId exists already when we want to insert a new record, we must first set the existing value to false (True = 0), then insert the new record. This means that we run an update and an insert for an soft update transaction. Because we cannot allow one part of this to succeed while the other part fails (such as the update part succeeding, while the insert fails), we wrap this in 1 transaction with a begin transaction.

The below query shows our two scenarios: in situation 1 the ValveId doesn't exist, so we would run an INSERT after the check. In situation 2, we would first need to update the existing record before inserting the new record. It should be of note that we should only have one true distinct ValveId at a time - a duplicate in this table would be considered a ValveId where two or more of its records have True values of 1.

---- Situation 1: Returns true because the valveid exists
DECLARE @valveid VARCHAR(12) = 'newrecord111'
SELECT ValveId FROM newValveImport WHERE ValveId = @valveid AND True = 1
---- Situation 2: Returns false because the record exists
DECLARE @valveid VARCHAR(12) = '00a6f80450'
SELECT ValveId FROM newValveImport WHERE ValveId = @valveid AND True = 1

In our next step, we handle the logic for these two situations:

---- We will run this twice:
DECLARE @output INT = 90, @input INT = 100
DECLARE @valveid VARCHAR(12) = 'newrecord111'
BEGIN TRAN
IF EXISTS (SELECT ValveId FROM newValveImport WHERE ValveId = @valveid AND True = 1)
BEGIN
UPDATE newValveImport
SET True = 0
WHERE ValveId = @valveid
END
INSERT INTO newValveImport (ValveId, ValveOutput, ValveInput, MeasurementTime, ImportDate, ImportFlag)
---- The output, input, time and import values are contrived for example purposes
VALUES (@valveid, @output, @input, GETDATE(),GETDATE(), 0)
COMMIT TRAN
SELECT *
FROM newValveImport
WHERE ValveId = 'newrecord111'

Our result shows two records with a ValveId of newrecord111, and only one record is True while the other is false. Every time we add a new record, our logic checks if a True value of the ValveId exists, updates True to 0 (false) if a record exists, and adds the record. If we call the transaction again, we'll see 3 records with the same ValveId, 2 which have True values of 0 (false) and the latest record with the same ValveId with a True value of 1.

Should We Use Soft Transactions? Now that we see how soft transactions work and the required logic we'd need,