String manipulation is a process to generate another form of existing data in a way the business uses or displayed as results in the reports. Previous SQL string function articles, I have written, including SQL string functions for Data Munging and SQL Substring function overview discussed data preparation and data management tasks using built-in SQL Server string functions.
SQL server also has some built-in character functions that also allow usto manipulate and transform data. At the same time, it is important to examine dataset, explore data values and encode or decode the values, as necessary, to generate meaningful data. It is important to know how to navigate through missing values in our datasets, understanding impact on calculations, queries, reports and data-set preparation and coming up with techniques to avoid letting Null values ruin our resultsets.
What is a NULL value?Before we delve into how to navigate the potential minefield of datasets with missing values and avoid stepping on a Null, let’s first take a quick look at what a NULL is.
As defined by Wikipedia
Null(or NULL) is a special marker used inStructured Query Languageto indicate that a data value does not exist in the database. Introduced by the creator of therelationaldatabase model,E. F. Codd,SQLNull serves to fulfil the requirement that alltruerelational database management systems (RDBMS)support a representation of “missing information and inapplicable information”. Codd also introduced the use of the lowercaseGreekomega(ω) symbol to represent Null indatabase theory. In SQL,NULLis areserved wordused to identify this marker. … This should not be confused with a value of 0. A null value indicates a lack of a value ― a lack of a value is not the same thing as a value of zero in the same way that a lack of an answer is not the same thing as an answer of “no”.
Furthermore …
SQL null is a state, not a value. This usage is quite different from most programming languages, where null value of a reference means it is not pointing to any object.
SQL does provide some handy functionality forworking with your character data in your SQL queries that we’ll describe in detail
SQL Coalesce functionThe SQL Server Coalesce and IsNull functions are used to handle NULL values. During the expression evaluation process the NULL values are replaced with the user-defined value.
The SQL Coalesce function evaluates the arguments in order and always returns first non-null value from the defined argument list.
Syntax COALESCE ( expression [ 1…n ] ) Properties of the Coalesce SQL function Expressions must be of same data-type It can contain multiple expressions The SQL Coalesce function is a syntactic shortcut for the Case expression Always evaluates for an integer first, an integer followed by character expression yields integer as an output. Examples: SELECT COALESCE (NULL,'A','B') SELECT COALESCE (NULL,100,20,30,40) SELECT COALESCE (NULL,NULL,20,NULL,NULL) SELECT COALESCE (NULL,NULL,NULL,NULL,NULL,'Prashanth') SELECT COALESCE (NULL,NULL,NULL,NULL,1,'Prashanth') SELECT COALESCE (NULL,NULL,NULL,NULL,NULL,'Prashanth',1)
SQL Coalesce in a string concatenation operation
In the following example, we’re going to concatenate some values.But, again, it’s just a review to let you know what happens when we have a NULL value. So, let’s go ahead and execute the T-SQL.And we can see that we encounter a NULL value while processing string concatenation operation. SQL server simply returns NULL whenever it encounters NULL value.The result is not a combination of the firstname, null, andlast name.
SELECT firstName +' '+MiddleName+' '+ LastName FullName FROM Person.Person
Let us handle the NULL values using a function called COALESCE. Itallows handling the behavior of the NULL value.So, in this case, use the coalesce SQL function to replaceany middle name NULL values with a value ‘ ‘ (Char(13)-space).The SQL statement should still concatenate all three names,but no NULL values will show up in the output.We now see that the full name gets displayed with a space in the middle, for NULL values.In this way, it is possible to efficiently customize the column values.
SELECT firstName +' '+COALESCE(MiddleName,'') +' '+ LastNameFROM Person.Person
SQL Coalesce function and pivoting
The following example returns the concatenated non-null values from the table ‘state’. In some cases, you may need to assign the concatenated static values to a variable. In this case, the values of the city column are parsed using the Coalesce SQL function and concatenated within a single quote to prepare a string of values. The output is then further manipulated to remove the last character to fetch a valid string of input value.
DROP TABLE IF EXISTS STATE; CREATE TABLE STATE ( CITY VARCHAR(50), STATE VARCHAR(500)) INSERT INTO STATE VALUES('Appleton','WI'),('Milwaukee','WI'),('Madison','WI'),('Miami','Florida'),('Jacksonville','Florida') DECLARE @col nvarchar(MAX); SELECT @col = COALESCE(@col,'') +''''+CITY +''''+ ',' FROM dbo.STATE WHERE state = 'WI'; SELECT substring(@col,1,len(@col)-1)Output:
SELECT '('+substring(@col,1,len(@col)-1)+')'
Scalar user-defined function and SQL Coalesce function
A user-defined function is created to return a string specific to the provided input and then the output is grouped using a grouping clause. In the following example, the scalar valued function returns the concatenated string values separated by ‘,’ for a specified ‘City’ input. The following example returns an output where the state column is grouped and its cities values are concatenated and separated by a delimiter ‘,’ (comma). You can also user STRING_AGG if you’re using SQL Server 2017. You can refer more information with the article Top SQL string functions in SQL Server 2017
CREATE FUNCTION dbo.tfn_CoalesceConcat ( @state varchar(100) ) RETURNS NVARCHAR(MAX) AS BEGIN DECLARE @str NVARCHAR(MAX); SELECT @str = COALESCE(@str + ', ', '') + CITY FROM dbo.STATE WHERE state = @state ORDER BY state; RETURN (@str); END GOHere is how we call the function name dbo.tfn_CoalesceConcat in the select statement.
The output is a concatenated stream of values separated by a delimiter ‘,’
SELECT state, city = dbo.tfn_CoalesceConcat(state) FROM dbo.state GROUP BY state ORDER BY state;
Data validation using SQL Coalesce function In the following example, we are going to find the emergency e