Depending on the input values, scalar functions in SQL return a single value. Instead of working with sets of rows, these functions work with individual values.

Common Scalar Functions
LEN(): Returns the length of a string.
UPPER(): Converts a string to uppercase.
LOWER(): Converts a string to lowercase.
ROUND(): Rounds a number to a specified decimal place.
GETDATE(): Returns the current date and time.

Example Usage of Scalar Functions

1. Using LEN() Function
SELECT LEN('Hello World') AS StringLength;

2. Using UPPER() and LOWER() Functions
SELECT UPPER('hello') AS UpperCase, LOWER('WORLD') AS LowerCase;

Output

3. Using ROUND() Function
SELECT ROUND(123.456, 2) AS RoundedValue

4. Using GETDATE() Function
SELECT GETDATE() AS CurrentDateTime;

5. Using ABS() Function
SELECT ABS(-25) AS AbsoluteValue;

6. Using SQRT() Function
SELECT SQRT(49) AS SquareRoot;

7. Using SUBSTRING() Function
SELECT SUBSTRING('SQL Functions', 5, 9) AS SubstringResult;

8. Using REPLACE() Function
SELECT REPLACE('Hello SQL', 'SQL', 'World') AS ReplacedString;

Advanced Use of Scalar Functions
1. Combining Scalar Functions
SELECT UPPER(LEFT('advanced scalar functions', 8)) AS Result;

2. Using Scalar Functions in Computations
SELECT ROUND(AVG(Salary), 2) AS AverageSalary FROM Employees;

3. Formatting Dates Using Scalar Functions
SELECT FORMAT(GETDATE(), 'yyyy-MM-dd') AS FormattedDate;

4. Custom Scalar Function Example
CREATE FUNCTION dbo.Getfullname(@FirstName NVARCHAR(50),
                                @LastName  NVARCHAR(50))
returns NVARCHAR(100)
AS
  BEGIN
      RETURN ( @FirstName + ' ' + @LastName )
  END;

Usage
SELECT dbo.GetFullName('John', 'Doe') AS FullName;

Advantages of Scalar Functions

• Helps in data formatting and transformation.
• Improves code readability and maintainability.
• Enhances query flexibility with built-in SQL functions.

Scalar functions are essential for manipulating individual values in SQL queries.

HostForLIFE.eu SQL Server 2022 Hosting
HostForLIFE.eu is European Windows Hosting Provider which focuses on Windows Platform only. We deliver on-demand hosting solutions including Shared hosting, Reseller Hosting, Cloud Hosting, Dedicated Servers, and IT as a Service for companies of all sizes.

A roadmap that describes how a query will be run is called a SQL Execution Plan. It aids in SQL query analysis and optimization.

Execution Plan Types
Estimated Execution Plan: This illustrates how the query would function even if it were not run.
Actual Execution Plan: This displays runtime information along with the query's actual execution.

How to Get the Execution Plan?
Using SQL Server Management Studio (SSMS)
Estimated Execution Plan: Press Ctrl + L or go to Query > Display Estimated Execution Plan.

Actual Execution Plan: Press Ctrl + M or go to Query > Include Actual Execution Plan, then run the query.

Using T-SQL Commands
Estimated Execution Plan
SET SHOWPLAN_XML ON;
SELECT * FROM Users WHERE UserID = 1;
SET SHOWPLAN_XML OFF;

Actual Execution Plan
SET STATISTICS XML ON;
SELECT * FROM Users WHERE UserID = 1;
SET STATISTICS XML OFF;

Understanding Execution Plan Components

Tips to Optimize SQL Queries

Use Indexes: Create indexes on frequently used columns.
Avoid SELECT *: Retrieve only the required columns.
Optimize Joins: Prefer INNER JOIN over OUTER JOIN if possible.
Check Execution Plan: Avoid Table Scans and use Index Seeks.
Avoid Functions on Indexed Columns: Example: WHERE YEAR(DateColumn) = 2023 prevents index usage.

In the next part, we will dive deeper into SQL execution plans, covering advanced topics like operator costs, parallelism, query hints, and execution plan caching, helping you gain a more comprehensive understanding of how SQL Server processes queries efficiently. Stay tuned!

HostForLIFE.eu SQL Server 2022 Hosting
HostForLIFE.eu is European Windows Hosting Provider which focuses on Windows Platform only. We deliver on-demand hosting solutions including Shared hosting, Reseller Hosting, Cloud Hosting, Dedicated Servers, and IT as a Service for companies of all sizes.

Statement of the Problem
Numerous database developers run into unforeseen inconsistencies while using SQL Server for calculations.  When the same mathematical phrase is evaluated differently, one typical problem occurs.  Take the following SQL Server code snippet, for example:

DECLARE @Number1 AS DECIMAL(26,7) = 0.9009000;
DECLARE @Number2 AS DECIMAL(26,7) = 1.000000000;
DECLARE @Number3 AS DECIMAL(26,7) = 1000.00000000;
DECLARE @Result  AS DECIMAL(26,7);

SET @Result = (@Number1 * @Number2) / @Number3;
SELECT @Result; -- 0.0009000
SET @Result = (@Number1 * @Number2);
SET @Result = (@Result / @Number3);
SELECT @Result; -- 0.0009009

In the first case, the output is 0.0009000, while in the second case, the output is 0.0009009. This divergence raises the question: Why are the results different when the same calculation is performed?

Explanation. Single Step Calculation
In the first approach, the entire expression (@Number1 * @Number2) / @Number3 is computed in a single step:

SQL Server first computes the product of @Number1 and @Number2, which equals 0.9009000.
Next, it divides that result by @Number3 (1000.00000000).

The result of this division is affected by how SQL Server handles precision and rounding for decimal operations. This might introduce slight inaccuracies, leading to the outcome of 0.0009000.

Multiple Step Calculation
In the second approach, the operations are separated into two distinct steps:

First, the calculation @Number1 * @Number2 is executed and stored in @Result. This retains the value of 0.9009000.
Then, the variable @Result is divided by @Number3 in a separate statement.

This step-by-step division allows SQL Server to apply different rounding and precision rules, which can sometimes yield a more accurate result of 0.0009009.

Conclusion
The difference in outputs can often be attributed to the varying treatment of precision and rounding during calculations:

• In a single-step calculation, SQL Server evaluates the entire expression at once, potentially altering precision during the process.
• In a multiple-step calculation, SQL Server retains more precision through intermediate results, leading to a different output.

Resolution
To achieve consistent results in SQL Server calculations, developers should consider controlling precision explicitly. For example, applying rounding can help standardize outcomes:
SET @Result = ROUND((@Number1 * @Number2) / @Number3, 7);

By managing precision and rounding explicitly, programmers can avoid discrepancies and ensure that their numerical calculations yield the expected results. Understanding these nuances in SQL Server can lead to more reliable and accurate database operations.

HostForLIFE.eu SQL Server 2022 Hosting
HostForLIFE.eu is European Windows Hosting Provider which focuses on Windows Platform only. We deliver on-demand hosting solutions including Shared hosting, Reseller Hosting, Cloud Hosting, Dedicated Servers, and IT as a Service for companies of all sizes.

Performance is frequently the main issue in any SQL Server system when working with big datasets. The INSERT operation is one frequent operation that occasionally turns into a bottleneck. The time required to input data increases with its size, which can have a major effect on system performance and user experience in general. The usage of the TABLOCK hint is one of the many methods and improvements that SQL Server offers to help speed up data insertions. When working with huge datasets or when parallelism is crucial, this straightforward yet effective method can significantly increase the pace of your INSERT operations.

What is the TABLOCK Hint?
The TABLOCK hint is a table-level lock hint that forces SQL Server to take a schema modification (Sch-M) lock on the target table when performing an INSERT, UPDATE, or DELETE operation. This hint ensures that the table is locked for the duration of the operation, which can help speed up data loading by minimizing logging and reducing contention.

A key benefit of the TABLOCK hint is that it reduces the amount of log space used during the operation, as the minimal logging mechanism is activated. This means that SQL Server does not have to log each individual row insertion, but rather just the metadata for the bulk operation. As a result, this significantly reduces the overhead and speeds up data loading.

Additionally, because the table is locked at the schema level, it allows SQL Server to parallelize the operation, leading to faster execution times. This is particularly useful for large-scale data-loading tasks.

When to Use TABLOCK Hint
The TABLOCK hint is ideal for scenarios where:

• You are inserting a large number of rows into a table.
• You can afford to lock the table for the duration of the operation (i.e., no other transactions need access to the table while the insert is in progress).
• You want to reduce the logging overhead and speed up bulk insertions.
• You want to use parallel insertions to take advantage of SQL Server's ability to use multiple threads for data loading.

It’s also important to note that the TABLOCK hint works well with temporary tables, so you can take advantage of these performance benefits when working with temp tables, often used in ETL processes or batch operations.

Benefits of Using TABLOCK

• Improved Performance: The primary benefit of using the TABLOCK hint is the performance improvement during large INSERT operations. By reducing the amount of logging, SQL Server can insert rows much faster.
• Parallel Insertion: With TABLOCK, SQL Server can use parallelism to load the data, speeding up the operation of systems with sufficient resources.
• Reduced Logging Overhead: Since SQL Server logs less information, the system consumes less log space, which can be crucial when working with large datasets.
• Works with Temp Tables: You can apply TABLOCK to temporary tables as well, giving you the same performance benefits for in-memory operations.

Example
Let’s consider a scenario where we need to insert a large number of rows from the Sales.SalesOrderDetail table into the Sales.SalesOrderDetailTemp table in the HostForLIFE database.
Create table script for Sales.SalesOrderDetailTem
USE [HostForLIFE]
GO

DROP TABLE IF EXISTS [Sales].[SalesOrderDetailTemp]
GO

SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO

CREATE TABLE [Sales].[SalesOrderDetailTemp](
    [SalesOrderID] [int] NOT NULL,
    [SalesOrderDetailID] [int] NOT NULL,
    [CarrierTrackingNumber] [nvarchar](25) NULL,
    [OrderQty] [smallint] NOT NULL,
    [ProductID] [int] NOT NULL,
    [SpecialOfferID] [int] NOT NULL,
    [UnitPrice] [money] NOT NULL,
    [UnitPriceDiscount] [money] NOT NULL,
    [LineTotal]  [money] NULL,
    [rowguid] [uniqueidentifier] ROWGUIDCOL  NOT NULL,
    [ModifiedDate] [datetime] NOT NULL,
 CONSTRAINT [PK_SalesOrderDetailTemp_SalesOrderID_SalesOrderDetailTempID] PRIMARY KEY CLUSTERED
(
    [SalesOrderID] ASC,
    [SalesOrderDetailID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY]
GO

Without the TABLOCK hint, this operation may take a considerable amount of time, especially when the table is large and the database is under load.

Here’s a basic example of how you can speed up the INSERT operation by using the TABLOCK hint.
USE HostForLIFE
GO

SET STATISTICS TIME, IO ON

SET NOCOUNT ON

INSERT INTO Sales.SalesOrderDetailTemp
SELECT *
FROM Sales.SalesOrderDetail;

Truncate the table.
USE HostForLIFE
GO

TRUNCATE TABLE Sales.SalesOrderDetailTemp

Now, let’s modify the query to use the TABLOCK hint.
USE HostForLIFE
GO

SET STATISTICS TIME, IO ON

SET NOCOUNT ON

INSERT INTO Sales.SalesOrderDetailTemp
WITH (TABLOCK)
SELECT *
FROM Sales.SalesOrderDetail;

Comparison
Execution 1 (without TABLOCK) took longer, with higher CPU and elapsed time (204 ms and 284 ms), indicating a slower operation. Execution 2 (with TABLOCK) performed better, completing in 125 ms CPU time and 157 ms elapsed time, making the TABLOCK version more efficient in this case.

Considerations When Using TABLOCK
While the TABLOCK hint can greatly improve performance, it’s important to be aware of some considerations:

• Table Locking: The TABLOCK hint locks the entire table for the duration of the operation. This means that other transactions cannot access the table until the INSERT operation is complete, so be sure that this behavior aligns with your application’s requirements.
• Transaction Log Growth: Although TABLOCK reduces the amount of logging, it still logs certain details of the operation. If you’re inserting a massive amount of data, you may need to monitor transaction log growth and ensure that you have enough log space available.
• Not Suitable for OLTP Workloads: The TABLOCK hint is more suited to batch operations or bulk-loading scenarios. It may not be appropriate for transactional systems that require frequent concurrent access to the table.

Conclusion
If you are working with large datasets and want to speed up your INSERT operations in SQL Server, the TABLOCK hint can be a game-changer. By reducing logging overhead and enabling parallel insertions, it helps improve performance and can significantly reduce the time it takes to load data.

HostForLIFE.eu SQL Server 2022 Hosting
HostForLIFE.eu is European Windows Hosting Provider which focuses on Windows Platform only. We deliver on-demand hosting solutions including Shared hosting, Reseller Hosting, Cloud Hosting, Dedicated Servers, and IT as a Service for companies of all sizes.

To effectively store NULL values while consuming the least amount of storage space, SQL Server offers Sparse Columns. When NULL values appear in a column in a sizable portion of rows, sparse columns are the best option.

1. What Are Sparse Columns?
Sparse columns are ordinary columns optimized for NULL storage. When a column is declared as SPARSE, it does not consume storage for NULL values, making them beneficial when a large number of rows have NULLs.

• Benefits of Sparse Columns.
• Saves storage by not allocating space for NULL values.
• Reduces I/O operations and improves performance for sparse datasets.
• Supports filtered indexes for better query performance.
• Drawbacks of Sparse Columns.
• Non-NULL values take up more space than regular columns.
• It cannot be used with.
• Text, Ntext, Image, Timestamp.
• User-defined data types.
• Computed columns.
• Default values (unless explicitly specified in an insert).
• CHECK constraints (except NULL constraints).

2. Declaring Sparse Columns
To use sparse columns, declare them with the SPARSE attribute.
Example. Creating a Table with Sparse Columns.
CREATE TABLE Employees (
EmployeeID INT PRIMARY KEY,
Name VARCHAR(100) NOT NULL,
PhoneNumber VARCHAR(20) SPARSE NULL,
Address NVARCHAR(255) SPARSE NULL
);

PhoneNumber and Address will not consume storage when NULL.

When storing non-NULL values, they use more storage than regular columns.

3. Storage Considerations
The impact on storage depends on the data type.

• For NULL values: Storage savings are significant.
• For Non-NULL values: Sparse columns require an additional 4 bytes.

When to Use Sparse Columns?

• When at least 20-40% of values are NULL, sparse columns save space.
• If NULLs are less frequent, regular columns are more efficient.

Example of Storage Cost for INT Data Type.

4. Using Sparse Columns with Column Sets
SQL Server provides Column Sets to handle sparse columns dynamically.

Example. Using Column Set for Dynamic Queries.
CREATE TABLE EmployeeData (
    EmployeeID INT PRIMARY KEY,
    Name VARCHAR(100) NOT NULL,
    PhoneNumber VARCHAR(20) SPARSE NULL,
    Address NVARCHAR(255) SPARSE NULL,
    AdditionalData XML COLUMN_SET FOR ALL_SPARSE_COLUMNS
);

AdditionalData (XML) aggregates all sparse column values into a single XML column dynamically.

Retrieving Data Using Column Set
SELECT EmployeeID, AdditionalData FROM EmployeeData;

The Column Set simplifies handling dynamic attributes.

5. Querying Sparse Columns Efficiently
Use Filtered Indexes to optimize queries on sparse columns.

Example. Creating a Filtered Index.
CREATE INDEX IX_Employees_PhoneNumber
ON Employees(PhoneNumber)
WHERE PhoneNumber IS NOT NULL;

This improves query performance for non-NULL sparse column searches.

Example. Query with Index Utilization.
SELECT Name, PhoneNumber
FROM Employees
WHERE PhoneNumber IS NOT NULL;

The filtered index ensures efficient lookups.

6. Checking Sparse Column Storage Space
You can analyze storage savings using sys.dm_db_index_physical_stats.

Check Space Savings.
SELECT name, is_sparse, max_length
FROM sys.columns
WHERE object_id = OBJECT_ID('Employees');

This shows which columns are SPARSE.

7. When NOT to Use Sparse Columns

Avoid sparse columns when:

• NULL values are less than 20-40% of total rows.
• The column is part of frequent aggregations.
• Additional 4-byte overhead is unacceptable.

8. Test Tables with sparse and without parse columns
Create two tables as below:

Add random data in both tables.

Check Table space.

In SQL Server, sparse columns are an effective technique to maximize NULL storage, minimize space consumption, and enhance performance. They function best when a large portion of the values are NULL and can be effectively queried with column sets and filtered indexes.

HostForLIFE.eu SQL Server 2022 Hosting
HostForLIFE.eu is European Windows Hosting Provider which focuses on Windows Platform only. We deliver on-demand hosting solutions including Shared hosting, Reseller Hosting, Cloud Hosting, Dedicated Servers, and IT as a Service for companies of all sizes.

In this article, we will learn how to create derived columns. Derived column task in SSIS is used to,

• Create new column
• Update existing columns
• Merge different columns into one column. For example, businesses want to concatenate first name, middle name, and last name to make it a full name.

Now let’s understand this with an example.
We will use the database, which has already been loaded into the SQL server, to create a derived column function.

Now let’s go to SSMS and see the table how it looks right now.

So here let’s say, the business wants to merge the first name, middle name, and last name of the customers in one column, so to perform this operation we need to use the derived column task. Now let’s go on the Data Flow Task in SSIS and perform the below steps.

Step 1. Create OLE DB Source connection – we will add OLE DB Source and Derived Column Task in the Data Flow Task and establish a source connection. We can see this in the screenshot below.

Step 2. Derived Column Transformation –To create new column values, we perform derived column transformation by applying expressions. We will go on Derived Column Editor to add derived column names and specify expressions to create new column values.

Step 3. Now click Ok and Create an OLE DB Destination connection – We will establish a new OLE DB Destination connection in the editor to push new data. So here will give the connection manager name and the new table name as CustomerNameDerivedCol.

Step 4. Now hit ok and go to Mappings to see available input columns and available destination columns, here you can notice that the same column name is mapped.

Step 5. Now the destination connection is established and the SSIS package is executed successfully.

Step 6. Now let’s verify this in SQL Server Management Studio (SSMS). In the below screenshot at the end of the result, we can see the CustomerName column. That’s how we derive a new column.

Summary
In this article, you have learned how to create the derived column in SSIS, hope you liked it. Looking forward to your comments and suggestions in the section below.

HostForLIFE.eu SQL Server 2022 Hosting
HostForLIFE.eu is European Windows Hosting Provider which focuses on Windows Platform only. We deliver on-demand hosting solutions including Shared hosting, Reseller Hosting, Cloud Hosting, Dedicated Servers, and IT as a Service for companies of all sizes.

In SQL Server, a transaction is a sequential set of actions (such statements or queries) carried out as a single task. Data in a database may be read, written, updated, or deleted throughout these activities. Transactions that adhere to the ACID characteristics guarantee data integrity.

• Atomicity: Ensures that a transaction is treated as a single unit. Either all operations in the transaction are completed successfully, or none are applied, maintaining the "all-or-nothing" principle.
• Consistency: Guarantees that a transaction transforms the database from one valid state to another, adhering to all defined rules, constraints, and relationships.
• Isolation: Ensures that concurrent transactions do not interfere with each other, maintaining data integrity as if transactions were executed sequentially.
• Durability: Ensures that once a transaction is committed, its changes are permanently recorded in the database, even in the event of a system failure.

Transaction Control
The following are the commands used to control transactions.

• BEGIN TRANSACTION: Marks the start of a transaction. All subsequent operations will be part of this transaction until it is committed or rolled back.
• COMMIT: Saves all the changes made during the transaction permanently to the database. Once committed, the changes cannot be undone.
• ROLLBACK: Reverts all changes made during the transaction to their state at the start of the transaction, effectively canceling the transaction.
• SAVEPOINT: Creates a checkpoint within a transaction. This allows rolling back a transaction to a specific point without undoing the entire transaction.
• RELEASE SAVEPOINT: Deletes a previously defined SAVEPOINT. Once released, SAVEPOINT can no longer be used for rollback.
• SET TRANSACTION: Configures a transaction with specific properties, such as setting it to read-only or read/write, or associating it with a specific rollback segment.

Types of Transactions

• Implicit Transactions
  
  • Automatically initiated by the database system when specific commands (e.g., INSERT, DELETE, UPDATE) are executed.
  • The transaction remains active until explicitly committed or rolled back by the user.
• Explicit Transactions
  
  • Manually initiated and controlled by the user.
  • Typically defined using BEGIN TRANSACTION, followed by COMMIT or ROLLBACK to either save or undo changes.
• Autocommit Transactions
  • The default transaction mode in most SQL systems.
  • Each individual SQL statement is automatically committed if it executes successfully. No explicit commands are needed to commit or rollback.
• Savepoints
  • Checkpoints within a transaction that allow partial rollbacks.
  • Useful for rolling back a specific part of a transaction without undoing the entire

Basic Transaction Syntax
Explicit Transaction Example
BEGIN TRANSACTION;

-- Deduct from one account
UPDATE EmpSalary_int
SET Salary = Salary - 100
WHERE EmpID = 1;

-- Add to another account
UPDATE EmpSalary_int
SET Salary = Salary + 100
WHERE EmpID = 2;

-- Commit the transaction
COMMIT;

Using ROLLBACK
BEGIN TRANSACTION;

UPDATE products
SET stock_quantity = stock_quantity - 10
WHERE product_id = 5;

-- Simulating an error
IF @@ERROR <> 0
BEGIN
    ROLLBACK;
    PRINT 'Transaction failed and was rolled back.';
END
ELSE
BEGIN
    COMMIT;
    PRINT 'Transaction completed successfully.';
END

Failed transaction

Savepoints for Partial Rollbacks
BEGIN TRANSACTION;

-- Step 1
INSERT INTO orders (order_id, customer_id, order_date)
VALUES (101, 1, GETDATE());

SAVE TRANSACTION SavePoint1;

-- Step 2
INSERT INTO order_details (order_id, product_id, quantity)
VALUES (101, 2, 5);

-- Rollback to SavePoint1 if needed
ROLLBACK TRANSACTION SavePoint1;

-- Commit remaining operations
COMMIT;

Here, we can see in the second table that order_details data is not saved because we have set rollback savepoint1.

TRY...CATCH Example
BEGIN TRY
    BEGIN TRANSACTION;

    UPDATE EmpSalary_int
    SET Salary = Salary - 100
    WHERE EmpID = 1;

    UPDATE EmpSalary_int
    SET Salary = Salary + 'null'
    WHERE EmpID= 2;

    COMMIT;
    PRINT 'Transaction completed successfully.';
END TRY
BEGIN CATCH
    ROLLBACK;
    PRINT 'An error occurred. Transaction rolled back.';
END CATCH;

Failed transaction

Conclusion
In SQL, transactions are sequences of operations performed as a single logical unit of work, ensuring data consistency and integrity. A transaction follows the ACID properties: Atomicity (all-or-nothing execution), Consistency (ensures data validity), Isolation (independence of concurrent transactions), and Durability (changes persist after completion). Transactions are crucial for managing database operations reliably and are typically controlled with commands like BEGIN, COMMIT, and ROLLBACK.

HostForLIFE.eu SQL Server 2022 Hosting
HostForLIFE.eu is European Windows Hosting Provider which focuses on Windows Platform only. We deliver on-demand hosting solutions including Shared hosting, Reseller Hosting, Cloud Hosting, Dedicated Servers, and IT as a Service for companies of all sizes.

To effectively handle massive volumes of data in a single call, a Table-Valued Parameter (TVP) in SQL Server allows an entire table to be passed as a parameter to a stored procedure or function. When sending many rows of data from an application to SQL Server, TVPs are commonly used.

Advantages of TVPs

• Increased effectiveness: Decreases the requirement for looping and multiple calls when passing numerous rows.
• Streamlined Code: Enhances code clarity by combining multiple inserts/updates into a single batch operation.
• Enhanced Performance: Reduces trips to the server, leading to substantial performance improvements with large datasets.

To achieve this Functionality Follow the Following Steps

Step 1. In SQL Server, a User-Defined Table Type (UDTT) needs to be created to define the structure (columns and data types) of the table parameter before using a TVP.

CREATE TYPE TVPUsersDetail AS TABLE (
    UserID INT,
    UserName NVARCHAR(50) NOT NULL,
    UserAge INT NOT NULL,
    UserAddress NVARCHAR(255)
);

Step 2. Create a physical table to store information like the one below.

CREATE TABLE UsersPrimaryTable (
    UserID INT,
    UserName NVARCHAR(50) NOT NULL,
    UserAge INT NOT NULL,
    UserAddress NVARCHAR(255)
);

Step 3. After defining the UDTT, create a stored procedure that takes a parameter of this table type.
CREATE PROCEDURE prcInsertDataIntoTVPTableData
    @MTVPUsersDetailsParam TVPUsersDetail READONLY
AS
BEGIN
    INSERT INTO TVPUsersDetail (UserID, UserName, UserAge, UserAddress)
    SELECT UserID, UserName, UserAge, UserAddress
    FROM @MTVPUsersDetailsParam;
END;

Step 4. Develop a sample application similar to the one provided below. I am utilizing WPF, but you may choose your own framework based on your specific needs.

UI View
<Window x:Class="TVPExample.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
        xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
        xmlns:local="clr-namespace:TVPExample"
        mc:Ignorable="d"
        Title="MainWindow" Height="450" Width="800">

    <StackPanel Margin="10">
        <!-- User ID Field -->
        <StackPanel Orientation="Horizontal" Margin="5">
            <TextBlock Text="User ID:" Width="100" VerticalAlignment="Center"/>
            <TextBox Name="UserIDTextBox" Width="200" Margin="5,0,0,0"/>
        </StackPanel>

        <!-- User Name Field -->
        <StackPanel Orientation="Horizontal" Margin="5">
            <TextBlock Text="User Name:" Width="100" VerticalAlignment="Center"/>
            <TextBox Name="UserNameTextBox" Width="200" Margin="5,0,0,0"/>
        </StackPanel>

        <!-- User Age Field -->
        <StackPanel Orientation="Horizontal" Margin="5">
            <TextBlock Text="User Age:" Width="100" VerticalAlignment="Center"/>
            <TextBox Name="UserAgeTextBox" Width="200" Margin="5,0,0,0"/>
        </StackPanel>

        <!-- User Address Field -->
        <StackPanel Orientation="Horizontal" Margin="5">
            <TextBlock Text="User Address:" Width="100" VerticalAlignment="Center"/>
            <TextBox Name="UserAddressTextBox" Width="200" Margin="5,0,0,0"/>
        </StackPanel>

        <!-- Buttons -->
        <Button Content="Add to List" Click="AddToList_Click" Margin="5"/>
        <Button Content="Submit to Database" Click="SubmitToDatabase_Click" Margin="5"/>

        <!-- DataGrid to Display Entries -->
        <DataGrid Name="UserDataGridView" Margin="5" Height="200"/>
    </StackPanel>
</Window>

Code behind Implementation
using System.Collections.ObjectModel;
using System.Data;
using System.Data.SqlClient;
using System.Windows;

namespace TVPExample
{
    /// <summary>
    /// Interaction logic for MainWindow.xaml
    /// </summary>
    public partial class MainWindow : Window
    {
        private ObservableCollection<UserDetailsModel> userCollection = new ObservableCollection<UserDetailsModel>();

        public MainWindow()
        {
            InitializeComponent();
            UserDataGridView.ItemsSource = userCollection;
        }

        string connectionString = @"Server=DESKTOP-JNM9BF1\SANJAYSERVER;Database=Demo;User Id=sa;Password=1234;"; // Change this connection string with your own

        private void AddToList_Click(object sender, RoutedEventArgs e)
        {
            if (int.TryParse(UserIDTextBox.Text, out int userId) &&
                int.TryParse(UserAgeTextBox.Text, out int userAge))
            {
                userCollection.Add(new UserDetailsModel
                {
                    UserID = userId,
                    UserName = UserNameTextBox.Text,
                    UserAge = userAge,
                    UserAddress = UserAddressTextBox.Text
                });

                // Clear input fields
                UserIDTextBox.Clear();
                UserNameTextBox.Clear();
                UserAgeTextBox.Clear();
                UserAddressTextBox.Clear();
            }
        }

        private void SubmitToDatabase_Click(object sender, RoutedEventArgs e)
        {
            var dataTable = new DataTable();
            dataTable.Columns.Add("UserID", typeof(int));
            dataTable.Columns.Add("UserName", typeof(string));
            dataTable.Columns.Add("UserAge", typeof(int));
            dataTable.Columns.Add("UserAddress", typeof(string));

            foreach (var user in userCollection)
            {
                dataTable.Rows.Add(user.UserID, user.UserName, user.UserAge, user.UserAddress);
            }

            InsertDataToDatabase(dataTable);
        }

        private void InsertDataToDatabase(DataTable dataTable)
        {
            try
            {
                using (var connection = new SqlConnection(connectionString))
                using (var command = new SqlCommand("prcInsertDataIntoTVPTableData", connection))
                {
                    command.CommandType = CommandType.StoredProcedure;

                    var parameter = command.Parameters.AddWithValue("@MTVPUsersDetailsParam", dataTable);
                    parameter.SqlDbType = SqlDbType.Structured;

                    connection.Open();
                    command.ExecuteNonQuery();
                }
            }
            catch (Exception ex)
            {
                throw;
            }
        }
    }
}

Model class
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace TVPExample
{
    internal class UserDetailsModel
    {
        public int UserID { get; set; }
        public string UserName { get; set; }
        public int UserAge { get; set; }
        public string UserAddress { get; set; }
    }
}

Application View

Explanation of the above Code

• Data Binding: UserDataGridView.ItemsSource = userCollection; binds the ObservableCollection<UserDetailsModel> to the DataGrid, making it easy to visualize added entries.
• Add to List: The AddToList_Click event reads values from TextBox inputs, converts them to the appropriate types, and adds a new UserDetailsModel instance to userCollection.After adding, it clears the TextBox fields.
• Submit to Database: The SubmitToDatabase_Click event creates a DataTable that matches the structure of the UserDetailsModel.It populates the DataTable from userCollection and sends it to SQL Server using a Table-Valued Parameter.
• Stored Procedure in SQL Server: The SQL Server stored procedure prcInsertDataIntoTVPTableData should be configured to accept a TVP with the structure of UserDetailsModel.

HostForLIFE.eu SQL Server 2022 Hosting
HostForLIFE.eu is European Windows Hosting Provider which focuses on Windows Platform only. We deliver on-demand hosting solutions including Shared hosting, Reseller Hosting, Cloud Hosting, Dedicated Servers, and IT as a Service for companies of all sizes.

I'll go over several techniques in this post for removing duplicate rows from a SQL Server table. The issues listed below will be covered in detail in this article.

• Introduction
• Various methods to remove duplicate rows from a table in SQL Server
• Points to Remember
• Conclusion

First of all, we are going to create a new database using the SQL Server. You can still do this step if you already have an existing database.

Create a new Database
The following SQL query creates a new database and a table. Copy this query and execute it in Query Explorer or the command line.
-- Execute the following query to create the database...
IF (DB_ID('Hostforlife_DeleteDuplicateRows') IS NOT NULL)
BEGIN
    USE master
    PRINT 'Database exists'
    DROP DATABASE Hostforlife_DeleteDuplicateRows
    PRINT 'Database Dropped...'
END
GO
CREATE DATABASE Hostforlife_DeleteDuplicateRows
PRINT 'New Database ''Hostforlife_DeleteDuplicateRows'' Created'
GO
USE [Hostforlife_DeleteDuplicateRows]
GO
-- Employee Table
CREATE TABLE [dbo].[Employee] (
    EmployeeID INT IDENTITY(31100, 1),
    EmployerID BIGINT NOT NULL DEFAULT 228866,
    FirstName VARCHAR(50) NOT NULL,
    LastName VARCHAR(50) NOT NULL,
    Email VARCHAR(255) NOT NULL,
    DepartmentID VARCHAR(100) NOT NULL,
    Age INT NOT NULL,
    GrossSalary BIGINT NOT NULL,
    PerformanceBonus BIGINT,
    ContactNo VARCHAR(25),
    PRIMARY KEY (EmployeeID)
);

Next, you can insert data to the tables using the SQL INSERT statement or by adding data directly to the tables in SSMS.

Let's check our table using the following query.

To get the data from the "Employee" table, use the following query.
SELECT * FROM Hostforlife_DeleteDuplicateRows..Employee

3 Ways to Delete Duplicate Rows From A Table In SQL Server
Here are 3 common methods that you can use to delete duplicate records from a table In SQL Server.

Method 1. Using GROUP BY and Having Clause
In this method, the SQL GROUP BY clause is used to identify and remove duplicate rows from a table.

Syntax
DELETE FROM <Table_Name>
WHERE ID NOT IN
(
    SELECT MAX(ID) AS MaxRecordID
    FROM <Table_Name>
    GROUP BY column1, columns2, ...
);

Example
DELETE FROM [Employee]
    WHERE EmployeeID NOT IN
    (
        SELECT MAX(EmployeeID) AS MaxRecordID
        FROM [Employee]
        GROUP BY [EmployerID], [FirstName], [LastName], [Email], [DepartmentID], [Age], [GrossSalary], [PerformanceBonus], [ContactNo]
    );

To verify the deletion, use the following query.

SELECT * FROM Hostforlife_DeleteDuplicateRows..Employee

Method 2. Using CTE (Common Table Expression)
CTE (Common Table Expression) can also be used to remove duplicate rows from a table in SQL Server.

Syntax
WITH CTE AS (
    SELECT
        column1,
        column2,
        ...
        ROW_NUMBER() OVER (
            PARTITION BY column1, column2, ...
            ORDER BY column1, column2, ...
        ) AS RowNumber
    FROM
        <Table_Name>
)
DELETE FROM CTE
WHERE RowNumber > 1;

Example
WITH CTE AS
(
    SELECT
        [EmployeeID],
        [EmployerID],
        [FirstName],
        [LastName],
        [Email],
        [DepartmentID],
        [Age],
        [GrossSalary],
        [PerformanceBonus],
        [ContactNo],
        ROW_NUMBER() OVER (
            PARTITION BY [EmployerID], [FirstName], [LastName], [Email], [DepartmentID], [Age], [GrossSalary], [PerformanceBonus], [ContactNo]
            ORDER BY [EmployerID], [FirstName], [LastName], [Email], [DepartmentID], [Age], [GrossSalary], [PerformanceBonus], [ContactNo]
        ) AS RowNumber
    FROM
        Employee
)
DELETE FROM CTE
WHERE RowNumber > 1;

To verify the deletion, use the following query.
SELECT * FROM Hostforlife_DeleteDuplicateRows..Employee

Method 3. Using Rank Function
The RANK function with PARTITION BY can also be used to remove duplicate rows from a table in SQL Server.

Syntax
DELETE E
    FROM <Table_Name> E
      INNER JOIN
    (
     SELECT *,
            RANK() OVER(PARTITION BY column1, column2, ...
            ORDER BY ID) rank
     FROM <Table_Name>
    ) T ON E.ID = t.ID
    WHERE rank > 1;

Example
DELETE E
FROM [Employee] E
INNER JOIN
(
    SELECT *,
           RANK() OVER (
               PARTITION BY [EmployerID], [FirstName], [LastName], [Email], [DepartmentID], [Age], [GrossSalary], [PerformanceBonus], [ContactNo]
               ORDER BY [EmployeeID]
           ) AS rank
    FROM [Employee]
) T ON E.[EmployeeID] = T.[EmployeeID]
WHERE rank > 1;

To verify the deletion, use the following query.
SELECT *
FROM Hostforlife_DeleteDuplicateRows..Employee;

Points to Remember
I would recommend you follow the points given below before deleting any type of record.

• Back up your data.
• Be sure to test your DELETE query with a SELECT statement.
• Choose an effective method as per the requirement to remove duplicate rows.

See you in the next article, till then, take care and be happy learning.

HostForLIFE.eu SQL Server 2022 Hosting
HostForLIFE.eu is European Windows Hosting Provider which focuses on Windows Platform only. We deliver on-demand hosting solutions including Shared hosting, Reseller Hosting, Cloud Hosting, Dedicated Servers, and IT as a Service for companies of all sizes.

Microsoft has included new features and improvements to SQL Server 2022 in order to boost functionality, usability, and speed. The DATE_BUCKET function is one of these new capabilities; it's a useful tool for developers and data professionals working with time-based data.  The DATE_BUCKET function streamlines interval management and date grouping, facilitating the aggregation and analysis of time-based data over predetermined timeframes. For SQL developers, data engineers, and DBAs who regularly handle temporal data analysis, this is especially helpful.

Understanding the DATE_BUCKET Function
What is DATE_BUCKET?
The DATE_BUCKET function in SQL Server 2022 is designed to help group and truncate dates into fixed intervals (or "buckets"). This makes it easier to group data for analysis over consistent time periods such as days, weeks, months, quarters, or years. This is especially useful for reporting and data visualization when you want to group time-based data into periods like hourly or weekly aggregates.

DATE_BUCKET (datepart, number, date [, origin ] )

--datepart: The part of the date you want to group by, such as a day, week, month, etc. This can be
--number: The size of the time bucket, which must be an integer. year, month, week, day, hour, minute, second, millisecond, etc.
--date: The date to be truncated and grouped by the interval and datepart.

Return Type
The function returns a date/time value that is rounded down to the start of the specified bucket. This allows for grouping time-based data into logical intervals.

How is DATE_BUCKET Different from Other T-SQL Functions?
Other T-SQL functions, like DATEADD, DATEDIFF, and DATEPART, are typically used to manipulate dates, extract parts of dates, or compute the difference between dates. However, these functions don't natively support the concept of fixed time intervals (buckets). DATE_BUCKET, on the other hand, allows for grouping dates into regular intervals, which can be critical for generating time-based reports.

Example 1. Month Interval Example
This example groups dates into 2-month intervals, starting from January 1, 2024.
DECLARE @DateOrigin date = '2024-01-01'
SELECT
    '1/2m' = DATE_BUCKET(MONTH, 2, CONVERT(date, '2024-01-01'), @DateOrigin),
    '1/2m' = DATE_BUCKET(MONTH, 2, CONVERT(date, '2024-02-01'), @DateOrigin),
    '2/2m' = DATE_BUCKET(MONTH, 2, CONVERT(date, '2024-03-01'), @DateOrigin),
    '2/2m' = DATE_BUCKET(MONTH, 2, CONVERT(date, '2024-04-01'), @DateOrigin),
    '1/2m' = DATE_BUCKET(MONTH, 2, CONVERT(date, '2024-05-01'), @DateOrigin),
    '1/2m' = DATE_BUCKET(MONTH, 2, CONVERT(date, '2024-06-01'), @DateOrigin),
    '2/2m' = DATE_BUCKET(MONTH, 2, CONVERT(date, '2024-07-01'), @DateOrigin),
    '2/2m' = DATE_BUCKET(MONTH, 2, CONVERT(date, '2024-08-01'), @DateOrigin)
GO

Output

Example 2. Week Interval Example
This example groups dates into 2-week intervals, starting from January 1, 2024.
DECLARE @DateOrigin date = '2024-01-01'
SELECT
    '1/2w' = DATE_BUCKET(WEEK, 2, CONVERT(date, '2024-01-01'), @DateOrigin),
    '1/2w' = DATE_BUCKET(WEEK, 2, CONVERT(date, '2024-01-08'), @DateOrigin),
    '2/2w' = DATE_BUCKET(WEEK, 2, CONVERT(date, '2024-01-15'), @DateOrigin),
    '2/2w' = DATE_BUCKET(WEEK, 2, CONVERT(date, '2024-01-22'), @DateOrigin),
    '1/2w' = DATE_BUCKET(WEEK, 2, CONVERT(date, '2024-01-29'), @DateOrigin),
    '1/2w' = DATE_BUCKET(WEEK, 2, CONVERT(date, '2024-02-05'), @DateOrigin),
    '2/2w' = DATE_BUCKET(WEEK, 2, CONVERT(date, '2024-02-12'), @DateOrigin),
    '2/2w' = DATE_BUCKET(WEEK, 2, CONVERT(date, '2024-02-19'), @DateOrigin)
GO

Output

Example 3. Day Interval Example
This example groups dates into 2-day intervals, starting from January 1, 2022.
DECLARE @DateOrigin date = '2024-01-01'
SELECT
    '1/2d' = DATE_BUCKET(DAY, 2, CONVERT(date, '2024-01-01'), @DateOrigin),
    '2/2d' = DATE_BUCKET(DAY, 2, CONVERT(date, '2024-01-02'), @DateOrigin),
    '1/2d' = DATE_BUCKET(DAY, 2, CONVERT(date, '2024-01-03'), @DateOrigin),
    '2/2d' = DATE_BUCKET(DAY, 2, CONVERT(date, '2024-01-04'), @DateOrigin),
    '1/2d' = DATE_BUCKET(DAY, 2, CONVERT(date, '2024-01-05'), @DateOrigin),
    '2/2d' = DATE_BUCKET(DAY, 2, CONVERT(date, '2024-01-06'), @DateOrigin),
    '1/2d' = DATE_BUCKET(DAY, 2, CONVERT(date, '2024-01-07'), @DateOrigin),
    '2/2d' = DATE_BUCKET(DAY, 2, CONVERT(date, '2024-01-08'), @DateOrigin)
GO

Output

Use Cases
1. Grouping Sales Data by Weekly Buckets
Suppose we want to analyze sales orders in AdventureWorks2022 and group the orders into weekly intervals. Using DATE_BUCKET, we can easily create these weekly buckets based on the OrderDate from the Sales.SalesOrderHeader table.

In this query
DATE_BUCKET(1, WEEK, OrderDate) groups the sales orders into weekly buckets, starting from the earliest OrderDate.
The query aggregates the total number of orders and the total sales (TotalDue) within each week.

USE AdventureWorks2022
GO

SELECT
    DATE_BUCKET(WEEK, 1, OrderDate) AS OrderWeek,
    COUNT(SalesOrderID) AS TotalOrders,
    SUM(TotalDue) AS TotalSales
FROM
    Sales.SalesOrderHeader
GROUP BY
    DATE_BUCKET(WEEK, 1, OrderDate)
ORDER BY
    OrderWeek

2. Monthly Sales Data Analysis
For longer-term trends, we may want to aggregate sales on a monthly basis. DATE_BUCKET makes it simple to group the data into months, just as easily as weeks.

Example: Monthly Sales Performance
This query aggregates the sales orders into monthly intervals using DATE_BUCKET(1, MONTH, OrderDate). You can easily visualize this data in a line graph or bar chart to track monthly sales performance over time.

USE AdventureWorks2022
GO

SELECT
    DATE_BUCKET(MONTH, 1, OrderDate) AS OrderMonth,
    COUNT(SalesOrderID) AS TotalOrders,
    SUM(TotalDue) AS TotalSales
FROM
    Sales.SalesOrderHeader
GROUP BY
    DATE_BUCKET(MONTH, 1, OrderDate)
ORDER BY
    OrderMonth

Output

3. Grouping Data in Custom Intervals (e.g., 10-Day Buckets)
While DATE_BUCKET allows for standard intervals like weeks or months, you can also group dates into custom intervals. For instance, if you want to create a report based on 10-day periods instead of full months or weeks, DATE_BUCKET can handle that too.

Here, we specify an interval of 10 days, and the sales orders are grouped into periods based on that interval. This can be useful in scenarios where typical calendar boundaries like weeks or months are too coarse or too fine.

USE AdventureWorks2022
GO

SELECT
    DATE_BUCKET(DAY, 10, OrderDate) AS OrderPeriod,
    COUNT(SalesOrderID) AS TotalOrders,
    SUM(TotalDue) AS TotalSales
FROM
    Sales.SalesOrderHeader
GROUP BY
    DATE_BUCKET(DAY, 10, OrderDate)
ORDER BY
    OrderPeriod

Output

Comparing DATE_BUCKET to Other Functions
1. DATEADD and DATEDIFF
In the past, SQL developers would use combinations of DATEADD and DATEDIFF to group dates into intervals. For example, you could group sales data by year with these functions:

While this method works, it is less intuitive and more cumbersome than using DATE_BUCKET, which allows for direct and flexible interval grouping.
USE AdventureWorks2022
GO

SELECT
    DATEADD(YEAR, DATEDIFF(YEAR, 0, OrderDate), 0) AS OrderYear,
    COUNT(SalesOrderID) AS TotalOrders,
    SUM(TotalDue) AS TotalSales
FROM
    Sales.SalesOrderHeader
GROUP BY
    DATEADD(YEAR, DATEDIFF(YEAR, 0, OrderDate), 0)

Output

2. FLOOR or CEILING on Date Calculations
Another workaround for bucketing dates was using FLOOR or CEILING in conjunction with date calculations. While effective, this approach was error-prone and harder to maintain.

For example, to group dates into weekly intervals using FLOOR, you might write something like this:

This code is not as readable as using DATE_BUCKET. The DATE_BUCKET function simplifies and abstracts away the complexity, making it easier to reason about your queries.
USE AdventureWorks2022
GO

SELECT
    FLOOR(DATEDIFF(DAY, '1900-01-01', OrderDate) / 7) AS WeekNumber,
    COUNT(SalesOrderID) AS TotalOrders
FROM
    Sales.SalesOrderHeader
GROUP BY
    FLOOR(DATEDIFF(DAY, '1900-01-01', OrderDate) / 7)

Output

Benefits of Using DATE_BUCKET

1. Simplified Code
One of the most apparent benefits of DATE_BUCKET is the simplification of code when compared to older methods of date bucketing. Instead of using complex expressions with DATEADD and DATEDIFF, you can now achieve the same result with a single, readable function.

2. Flexibility and Power
DATE_BUCKET provides a powerful tool for aggregating time-based data in flexible ways. Whether you need to group data by week, month, or even custom intervals like ten days or 15 minutes, DATE_BUCKET makes it easy to express and execute these groupings.

3. Improved Performance
By natively supporting time-based intervals in a straightforward function, DATE_BUCKET improves performance over workarounds that rely on complex date manipulation functions such as combinations of DATEADD, DATEDIFF, and FLOOR. These traditional approaches often require multiple calculations and transformations to achieve similar results, which can increase both complexity and computational overhead.

Conclusion
The introduction of the DATE_BUCKET function in SQL Server 2022 marks a significant enhancement for SQL developers, data engineers, and DBAs who frequently work with time-based data. By simplifying the process of grouping dates into consistent intervals, DATE_BUCKET not only makes queries more readable and easier to maintain but also improves performance by reducing reliance on complex, manual date manipulation functions. With its ability to streamline queries, improve code maintainability, and optimize performance, DATE_BUCKET represents a valuable addition to the SQL Server toolkit, empowering professionals to better manage and analyze their time-series data.

HostForLIFE.eu SQL Server 2022 Hosting
HostForLIFE.eu is European Windows Hosting Provider which focuses on Windows Platform only. We deliver on-demand hosting solutions including Shared hosting, Reseller Hosting, Cloud Hosting, Dedicated Servers, and IT as a Service for companies of all sizes.