AskOverflow.Dev

AskOverflow.Dev Logo AskOverflow.Dev Logo

AskOverflow.Dev Navigation

  • 主页
  • 系统&网络
  • Ubuntu
  • Unix
  • DBA
  • Computer
  • Coding
  • LangChain

Mobile menu

Close
  • 主页
  • 系统&网络
    • 最新
    • 热门
    • 标签
  • Ubuntu
    • 最新
    • 热门
    • 标签
  • Unix
    • 最新
    • 标签
  • DBA
    • 最新
    • 标签
  • Computer
    • 最新
    • 标签
  • Coding
    • 最新
    • 标签
主页 / dba / 问题

问题[index-tuning](dba)

Martin Hope
Ross Bush
Asked: 2022-10-06 05:46:46 +0800 CST

基于键字段的聚集索引能否比大表的非聚集覆盖索引更高效

  • 0

我正在研究一个会变得非常大的 DW 报告表。为简单起见,我将显示该表如下:

BigTable
--------
TableID INT IDENTITY NOT NULL,
CompanyName NVARCHAR(100) NOT NULL

每个查询都将使用公司名称在数据分区(而不是物理分区)内进行查询。

由于该表可能包含超过十亿行,并且每个公司的数据分布非常均匀,因此按公司查询应该尽可能快。我正处于设置一些测试的阶段,但在这样做之前,我想我会问一下,看看这是否会浪费时间。

我的想法是确定如果每个公司的数据分区通过聚集索引在磁盘上彼此相邻放置,那么数据检索是否会比仅使用非聚集索引覆盖 CompanyName 更快。

示例 1:这是 IDENTITY 列是 PK 但不是 CLUSTERED 的变体。CompanayName 和 TableID 结合起来形成聚集索引,因此数据将按公司在磁盘上排序。

CREATE TABLE [dbo].[BigTable](
    [TableID] [int] IDENTITY(1,1) NOT NULL,
    [CompanyName] [nvarchar](100) NOT NULL,
 CONSTRAINT [PK_BigTable] PRIMARY KEY NONCLUSTERED 
(
    [TableID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 97, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY]

GO

CREATE UNIQUE CLUSTERED INDEX [CLUSTERED_ByCompanyName_TableID] ON [dbo].[BigTable]
(
    [CompanyName] ASC,
    [TableID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 97, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
GO

这是创建具有覆盖索引的表的传统方式。

CREATE TABLE [dbo].[BigTable](
    [TableID] [int] IDENTITY(1,1) NOT NULL,
    [CompanyName] [nvarchar](200) NOT NULL,
 CONSTRAINT [PK_BigTable] PRIMARY KEY CLUSTERED 
(
    [TableID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 97, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY]

GO

CREATE NONCLUSTERED INDEX [IX_ByCompanyName] ON [dbo].[BigTable]
(
    [CompanyName] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 97, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
GO

有谁知道在使用第一个示例而不是第二个示例时是否会有任何性能改进?

编辑:我倾向于在公司中使用聚集索引。如果行需要唯一引用,TableID 只是用作 PK 的自动增量字段。我觉得聚集索引搜索/扫描比索引扫描/搜索更快。

我希望您可以轻松地根据 companyid 之类的东西进行分区或分片。

基本查询的形式为

SELECT
    SUM(FieldA) OVER (PARTITION BY ...) a,
    COUNT(1) OVER (PARTITION BY...) b
    ...       
FROM
    BigTable 
WHERE
    CompanyName = 'NABISCO'
GROUP BY
   ....
ORDER BY
   ....
sql-server index-tuning
  • 1 个回答
  • 57 Views
Martin Hope
bigchief
Asked: 2022-10-04 04:17:06 +0800 CST

当表上有两个唯一索引且一个索引中的所有列也在另一个索引中时,是否存在性能损失?

  • 7

在我的数据库中,我有一个带有两个索引的表。出于数据完整性的原因,我需要在 a、b 列上有一个唯一索引 #1。我有另一个索引#2,列 c、a、b 用于性能原因。我注意到这个索引#2 也是独一无二的。

索引#2 的唯一性在我看来是多余的,因为你不能在索引#2 中有重复的值,而在索引#1 中也没有重复的值。我很想更改索引#2,因此它不再是唯一的,因为我想数据库引擎可能会对索引#2 中的 c、a、b 执行第二次检查,以确保每次插入一行时这些列的唯一性,导致即使永远不会有重复的值,也会影响性能。这个对吗?

有什么方法可以删除 a,b 上的索引 #1 并保留 c,a,b 上的索引 #2 但仍然对列 a,b 强制唯一约束而不维护两个单独的索引?这将允许我只有一个包含所有三列的索引,但仍然对 a、b 强制执行我的数据完整性约束。我不需要 a,b 上的索引来提高性能,因为我的所有选择查询都在 where 子句中包含 c 列。这会是唯一约束而不是索引的用例吗?我认为数据库引擎基本上将这两个构造视为相同(参见这篇文章:我何时应该使用唯一约束而不是唯一索引?)。

请记住,索引不是多余的,但索引的“唯一性”是多余的。似乎让索引#2 不唯一是一件轻而易举的事。但这会带来任何实际的性能提升吗?即使索引#1 中的列完全包含在索引#2 中,数据库是否会检查两个索引的唯一性?

一些答案询问了用于从该表中选择数据的示例查询。以下是最常见的:

Select [some other columns] from table where c=1 and a=2
Select [some other columns] from table where c=1
Select [some other columns] from table where c=1 and a=2 and b=3

这些查询通常包括选择许多不在任何索引中的其他列。

我们通常从不运行的是这样的查询:

Select [stuff] from table where a=2 and b=3
sql-server index-tuning
  • 3 个回答
  • 740 Views
Martin Hope
fmi21
Asked: 2022-06-27 00:40:46 +0800 CST

3 列的聚集索引是否太大?

  • 3

我的目标是设计一个表,可以通过外部 id( uniqueidentifier)、内部 id( bigint) 进行查询,始终与和/或与 ( , n=0,10) 结合使用companyId(bigint),这两个条件都起到所有权检查的作用。userId(bigint)dashboardId(bigint)dashboardId IN @0, ..., @n

我想出了以下指数组成:

CREATE CLUSTERED INDEX Mytable_createdBy_cix ON Mytable(companyId, createdBy, dashboardId)

CREATE UNIQUE NONCLUSTERED INDEX Mytable_extId_nix ON Mytable(extId) INCLUDE (valueD, valueN)

CREATE UNIQUE NONCLUSTERED INDEX Mytable_chartId_nix ON Mytable (chartId) INCLUDE (valueD, valueN)

我不知道以下问题的答案:

  • 聚集索引是否因为非唯一而坏?我应该添加隔离键而不使用自动分配的uniqueifier吗?
  • 3 * 8 字节列 + 4 字节唯一符(总共 28 字节)对于聚集索引来说太多了吗?我读到它包含在每个唯一非聚集索引的包含页面中(根据使用的键添加额外的 16 或 8 个字节)。
  • 这种索引设计对下面的查询是否有意义?

我计划运行查询,类似于:

SELECT chartId, valueD, valueN FROM Mytable WHERE companyId = @companyId AND createdBy = @userId

SELECT chartId, valueD, valueN FROM Mytable WHERE companyId = @companyId AND createdBy = @userId AND dashboardId = @dashboardId

SELECT chartId, valueD, valueN FROM Mytable WHERE dashboardId IN (@0, @1, @2)

SELECT chartId, valueD, valueN FROM Mytable WHERE (companyId = @companyId AND createdBy = @userId AND dashboardId = @dashboardId) OR dashboardId IN (@0, @1, @2)

UPDATE Mytable SET valueD = @valueD WHERE companyId = @companyId AND createdBy = @userId AND chartId = @chartId

UPDATE Mytable SET valueD = @valueD WHERE companyId = @companyId AND createdBy = @userId AND extChartId= @extId

UPDATE Mytable SET valueD = @valueD WHERE ((companyId = @companyId AND createdBy = @userId) OR dashboardId IN (@0, @1, @2)) AND extChartId= @extId

我确实知道,最好在 stackexchange 上提问时测试、评估执行计划并分享它们,但这是设计阶段,因此还没有实际的数据或表格。

我可以调整键/索引/表结构以更好地适应查询。我只是希望在第一次创建它们时至少部分正确,所以这个问题不会被重新讨论。

非常感谢您提前提供的任何帮助。

sql-server index-tuning
  • 2 个回答
  • 352 Views
Martin Hope
Antyan
Asked: 2022-06-16 06:38:44 +0800 CST

数据仓库和数据湖更新的最佳索引策略

  • 2

我们有一个 sql server 数据库,用作数据湖和数据仓库。数据库中的每个表都有一些标准化的定义,因为我们现在有 600 个左右的表,因此维护需要在某种程度上自动化。

加载每个表的一般过程是首先将表的副本加载到 changeLog 模式中的哈希表中(如果我们可以确定更改的记录是什么,有时只加载更改的记录),然后将 changeLog 表与目标表进行比较桌子。目标表用于报告,因此这种 changeLog 方法允许我们持久化目标表并仅应用最少的 UPDATE/INSERT 操作。

每个目标表都有一个唯一键/业务键,可通过配置表识别,并具有在每个表中命名相同的标准化审计列。审计专栏告诉我们

  1. 当记录被添加到数据仓库时
  2. 上次在数据仓库中更新记录的时间
  3. 记录是否已在源中删除
  4. 使用 HASHBYTES('SHA2_256', CONCAT()) 更改的记录标识符

更改后的记录标识符曾经是 CHECKSUM(),但我们发现 CHECKSUM() 的冲突率太高而无法信任。我刚刚在每个表中添加了一个 HASHBYTES() 列并填充了它。

我将 HASHBYTES() 列创建为 VARBINARY(MAX)。现在,每次加载表时,我们可以通过将在 changeLog 表中计算的新 HASHBYTES() 值与目标表中的持久值进行比较来判断是否需要更新记录。

我立即注意到从 INT CHECKSUM() 切换到 VARBINARY(MAX) HASHBYTES() 导致更新检查过程显着减慢。我在每个 CHECKSUM 列上都有 NONCLUSTERED 索引,但在我刚刚添加的 HASHBYTES 列上没有。每个表的唯一键也有聚集索引。

  1. 添加以检查更新的理想索引是什么?
  2. 我可以为每个表添加一个标准化索引吗?
  3. VARBINARY(MAX) 是正确的数据类型还是可以安全地将其缩小到更小的大小?

希望这足以让这个问题变得有意义。我需要尽快加快这个过程。

编辑:我正在添加一个大型 SQL 脚本作为示例,它具有表的 changeLog 版本和目标版本的示例表定义,以及为更新目标版本而运行的查询。

--OBJECT DEFINITIONS
CREATE TABLE [changeLog].[DimSalesOffice](
    [Sales Office Code] [varchar](3) NOT NULL,
    [CCN_Key] [uniqueidentifier] NOT NULL,
    [Sales Office Name (Short)] [nvarchar](14) NOT NULL,
    [Sales Office Name (Long)] [nvarchar](50) NOT NULL,
    [Sales Office City] [nvarchar](50) NOT NULL,
    [Sales Office StateProvince] [nvarchar](50) NOT NULL,
    [Sales Office Postal Code] [nvarchar](20) NOT NULL,
    [Sales Office Country Code] [varchar](3) NOT NULL,
    [Sales Office Address] [nvarchar](65) NOT NULL,
    [Sales Office Address (Line 2)] [nvarchar](65) NOT NULL,
    [Sales Office Name (Short - Native Language)] [nvarchar](14) NOT NULL,
    [Sales Office Name (Long - Native Language)] [nvarchar](50) NOT NULL,
    [Sales Office Address (Native Language)] [nvarchar](65) NOT NULL,
    [Sales Office Address (Line 2 - Native Language)] [nvarchar](65) NOT NULL,
    [Sales Office City (Native Language)] [nvarchar](100) NOT NULL,
    [Sales Office StateProvince (Native Language)] [nvarchar](100) NOT NULL,
    [Sales Office Region] [nvarchar](100) NULL,
    [Native Language Code] [varchar](2) NOT NULL
) ON [PRIMARY]
GO

CREATE TABLE [dbo].[DimSalesOffice](
    [SalesOffice_Key] [uniqueidentifier] NOT NULL,
    [Sales Office Code] [varchar](3) NOT NULL,
    [CCN_Key] [uniqueidentifier] NOT NULL,
    [Sales Office Name (Short)] [nvarchar](14) NOT NULL,
    [Sales Office Name (Long)] [nvarchar](50) NOT NULL,
    [Sales Office City] [nvarchar](50) NOT NULL,
    [Sales Office StateProvince] [nvarchar](50) NOT NULL,
    [Sales Office Postal Code] [nvarchar](20) NOT NULL,
    [Sales Office Country Code] [varchar](3) NOT NULL,
    [Sales Office Address] [nvarchar](65) NOT NULL,
    [Sales Office Address (Line 2)] [nvarchar](65) NOT NULL,
    [Sales Office Name (Short - Native Language)] [nvarchar](14) NOT NULL,
    [Sales Office Name (Long - Native Language)] [nvarchar](50) NOT NULL,
    [Sales Office Address (Native Language)] [nvarchar](65) NOT NULL,
    [Sales Office Address (Line 2 - Native Language)] [nvarchar](65) NOT NULL,
    [Sales Office City (Native Language)] [nvarchar](100) NOT NULL,
    [Sales Office StateProvince (Native Language)] [nvarchar](100) NOT NULL,
    [Sales Office Region] [nvarchar](100) NULL,
    [Native Language Code] [varchar](2) NOT NULL,
    [DW_CreatedOn] [datetime2](7) NULL,
    [DW_ModifiedOn] [datetime2](7) NULL,
    [DW_IsDeleted?] [bit] NULL,
    [DW_Checksum] [int] NULL,
    [Source_ModifiedOn] [datetime2](7) NULL,
    [DW_Hashbytes] [varbinary](max) NULL,
PRIMARY KEY NONCLUSTERED 
(
    [SalesOffice_Key] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO

ALTER TABLE [dbo].[DimSalesOffice] ADD  DEFAULT (newsequentialid()) FOR [SalesOffice_Key]
GO

CREATE UNIQUE CLUSTERED INDEX [IX_UK_DimSalesOffice] ON [dbo].[DimSalesOffice]
(
    [Sales Office Code] ASC
)
GO

--MERGE QUERY
DECLARE @InsertRecordCount INT, @UpdateRecordCount INT;

/*****UPDATE*****/
UPDATE [dbo].[DimSalesOffice] SET
    [CCN_Key] = [Source].[CCN_Key],
    [Sales Office Name (Short)] = [Source].[Sales Office Name (Short)],
    [Sales Office Name (Long)] = [Source].[Sales Office Name (Long)],
    [Sales Office City] = [Source].[Sales Office City],
    [Sales Office StateProvince] = [Source].[Sales Office StateProvince],
    [Sales Office Postal Code] = [Source].[Sales Office Postal Code],
    [Sales Office Country Code] = [Source].[Sales Office Country Code],
    [Sales Office Address] = [Source].[Sales Office Address],
    [Sales Office Address (Line 2)] = [Source].[Sales Office Address (Line 2)],
    [Sales Office Name (Short - Native Language)] = [Source].[Sales Office Name (Short - Native Language)],
    [Sales Office Name (Long - Native Language)] = [Source].[Sales Office Name (Long - Native Language)],
    [Sales Office Address (Native Language)] = [Source].[Sales Office Address (Native Language)],
    [Sales Office Address (Line 2 - Native Language)] = [Source].[Sales Office Address (Line 2 - Native Language)],
    [Sales Office City (Native Language)] = [Source].[Sales Office City (Native Language)],
    [Sales Office StateProvince (Native Language)] = [Source].[Sales Office StateProvince (Native Language)],
    [Sales Office Region] = [Source].[Sales Office Region],
    [Native Language Code] = [Source].[Native Language Code],
    [DW_Checksum] =
        CHECKSUM(
            [Source].[CCN_Key],
            [Source].[Sales Office Name (Short)],
            [Source].[Sales Office Name (Long)],
            [Source].[Sales Office City],
            [Source].[Sales Office StateProvince],
            [Source].[Sales Office Postal Code],
            [Source].[Sales Office Country Code],
            [Source].[Sales Office Address],
            [Source].[Sales Office Address (Line 2)],
            [Source].[Sales Office Name (Short - Native Language)],
            [Source].[Sales Office Name (Long - Native Language)],
            [Source].[Sales Office Address (Native Language)],
            [Source].[Sales Office Address (Line 2 - Native Language)],
            [Source].[Sales Office City (Native Language)],
            [Source].[Sales Office StateProvince (Native Language)],
            [Source].[Sales Office Region],
            [Source].[Native Language Code],
            0
        ),
    [DW_Hashbytes] = 
        HASHBYTES(
            'SHA2_256',
            ISNULL(CAST([Source].[CCN_Key] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Name (Short)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Name (Long)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office City] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office StateProvince] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Postal Code] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Country Code] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Address] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Address (Line 2)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Name (Short - Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Name (Long - Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Address (Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Address (Line 2 - Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office City (Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office StateProvince (Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Region] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Native Language Code] AS NVARCHAR(MAX)), '') + '|'
            + '0'
        ),
    [Source_ModifiedOn] = NULL,
    [DW_ModifiedOn] = GETUTCDATE(),
    [DW_IsDeleted?] = 0
FROM [changeLog].[DimSalesOffice] [Source]
JOIN [dbo].[DimSalesOffice]
    ON [Source].[Sales Office Code] = [DimSalesOffice].[Sales Office Code]
    AND ISNULL([DimSalesOffice].[DW_Hashbytes], HASHBYTES('SHA2_256', '')) <> HASHBYTES(
            'SHA2_256',
            ISNULL(CAST([Source].[CCN_Key] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Name (Short)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Name (Long)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office City] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office StateProvince] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Postal Code] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Country Code] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Address] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Address (Line 2)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Name (Short - Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Name (Long - Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Address (Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Address (Line 2 - Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office City (Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office StateProvince (Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Region] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Native Language Code] AS NVARCHAR(MAX)), '') + '|'
            + '0'
        )
SET @UpdateRecordCount = @@ROWCOUNT;

/*****Soft Deletes*****/
UPDATE [dbo].[DimSalesOffice] SET
    [DW_Checksum] = 0,
    [DW_Hashbytes] = 
        HASHBYTES(
            'SHA2_256',
            ISNULL(CAST([DimSalesOffice].[CCN_Key] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([DimSalesOffice].[Sales Office Name (Short)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([DimSalesOffice].[Sales Office Name (Long)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([DimSalesOffice].[Sales Office City] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([DimSalesOffice].[Sales Office StateProvince] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([DimSalesOffice].[Sales Office Postal Code] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([DimSalesOffice].[Sales Office Country Code] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([DimSalesOffice].[Sales Office Address] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([DimSalesOffice].[Sales Office Address (Line 2)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([DimSalesOffice].[Sales Office Name (Short - Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([DimSalesOffice].[Sales Office Name (Long - Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([DimSalesOffice].[Sales Office Address (Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([DimSalesOffice].[Sales Office Address (Line 2 - Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([DimSalesOffice].[Sales Office City (Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([DimSalesOffice].[Sales Office StateProvince (Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([DimSalesOffice].[Sales Office Region] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([DimSalesOffice].[Native Language Code] AS NVARCHAR(MAX)), '') + '|'
            + '1'
        ),
    [Source_ModifiedOn] = NULL,
    [DW_ModifiedOn] = GETUTCDATE(),
    [DW_IsDeleted?] = 1
FROM [dbo].[DimSalesOffice]
WHERE NOT EXISTS
    (
        SELECT 1
        FROM [changeLog].[DimSalesOffice] [Source]
        WHERE [Source].[Sales Office Code] = [DimSalesOffice].[Sales Office Code]
    )
SET @UpdateRecordCount = @UpdateRecordCount + @@ROWCOUNT;

/*****INSERT*****/
INSERT INTO [dbo].[DimSalesOffice]
    (
        [Sales Office Code],
        [CCN_Key],
        [Sales Office Name (Short)],
        [Sales Office Name (Long)],
        [Sales Office City],
        [Sales Office StateProvince],
        [Sales Office Postal Code],
        [Sales Office Country Code],
        [Sales Office Address],
        [Sales Office Address (Line 2)],
        [Sales Office Name (Short - Native Language)],
        [Sales Office Name (Long - Native Language)],
        [Sales Office Address (Native Language)],
        [Sales Office Address (Line 2 - Native Language)],
        [Sales Office City (Native Language)],
        [Sales Office StateProvince (Native Language)],
        [Sales Office Region],
        [Native Language Code],
        [DW_Checksum],
        [DW_Hashbytes],
        [Source_ModifiedOn],
        [DW_ModifiedOn],
        [DW_IsDeleted?],
        [DW_CreatedOn]
    )
SELECT
    [Source].[Sales Office Code],
    [Source].[CCN_Key],
    [Source].[Sales Office Name (Short)],
    [Source].[Sales Office Name (Long)],
    [Source].[Sales Office City],
    [Source].[Sales Office StateProvince],
    [Source].[Sales Office Postal Code],
    [Source].[Sales Office Country Code],
    [Source].[Sales Office Address],
    [Source].[Sales Office Address (Line 2)],
    [Source].[Sales Office Name (Short - Native Language)],
    [Source].[Sales Office Name (Long - Native Language)],
    [Source].[Sales Office Address (Native Language)],
    [Source].[Sales Office Address (Line 2 - Native Language)],
    [Source].[Sales Office City (Native Language)],
    [Source].[Sales Office StateProvince (Native Language)],
    [Source].[Sales Office Region],
    [Source].[Native Language Code],
    [DW_Checksum] = 
        CHECKSUM(
            [Source].[CCN_Key],
            [Source].[Sales Office Name (Short)],
            [Source].[Sales Office Name (Long)],
            [Source].[Sales Office City],
            [Source].[Sales Office StateProvince],
            [Source].[Sales Office Postal Code],
            [Source].[Sales Office Country Code],
            [Source].[Sales Office Address],
            [Source].[Sales Office Address (Line 2)],
            [Source].[Sales Office Name (Short - Native Language)],
            [Source].[Sales Office Name (Long - Native Language)],
            [Source].[Sales Office Address (Native Language)],
            [Source].[Sales Office Address (Line 2 - Native Language)],
            [Source].[Sales Office City (Native Language)],
            [Source].[Sales Office StateProvince (Native Language)],
            [Source].[Sales Office Region],
            [Source].[Native Language Code],
            0
        ),
    [DW_Hashbytes] = 
        HASHBYTES(
            'SHA2_256',
            ISNULL(CAST([Source].[CCN_Key] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Name (Short)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Name (Long)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office City] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office StateProvince] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Postal Code] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Country Code] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Address] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Address (Line 2)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Name (Short - Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Name (Long - Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Address (Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Address (Line 2 - Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office City (Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office StateProvince (Native Language)] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Sales Office Region] AS NVARCHAR(MAX)), '') + '|'
            + ISNULL(CAST([Source].[Native Language Code] AS NVARCHAR(MAX)), '') + '|'
            + '0'
        ),
    [Source_ModifiedOn] = NULL,
    [DW_ModifiedOn] = GETUTCDATE(),
    [DW_IsDeleted?] = 0,
    [DW_CreatedOn] = GETUTCDATE()
FROM [changeLog].[DimSalesOffice] [Source]
WHERE NOT EXISTS
    (
        SELECT 1
        FROM [dbo].[DimSalesOffice]
        WHERE [Source].[Sales Office Code] = [DimSalesOffice].[Sales Office Code]
    )
SET @InsertRecordCount = @@ROWCOUNT;

SELECT [Update Record Count] = @UpdateRecordCount, [Insert Record Count] = @InsertRecordCount;
sql-server index-tuning
  • 2 个回答
  • 99 Views
Martin Hope
Nullified
Asked: 2022-06-10 00:41:46 +0800 CST

在 postgresql 中使用表时删除索引的影响

  • 0

在使用表时删除或删除索引有什么影响?

样本:


从


表:order_tbl 索引:order_id_index(不是 btree)

至


需要将其更改为 index:order_id_index(btree)


删除索引时/之后是否有影响或数据丢失?

postgresql index-tuning
  • 1 个回答
  • 81 Views
Martin Hope
Mandelbrotter
Asked: 2022-01-12 08:43:46 +0800 CST

如果所有查询只需要完美匹配(没有部分文本搜索),应该使用什么索引

  • 0

我想提高在var dbObj = dbSet.Where(x => x.Name == name).FirstOrDefault();下面粘贴的表上完成的查询的性能。我正在考虑在[Name]. 如果我不需要查询子字符串的能力,我应该使用什么类型的索引?

CREATE TABLE [dbo].[TitleTypes] (
    [Id]   INT          IDENTITY (1, 1) NOT NULL,
    [Name] VARCHAR (MAX) NOT NULL,
    SysStartTime DATETIME2 GENERATED ALWAYS AS ROW START NOT NULL,
    SysEndTime DATETIME2 GENERATED ALWAYS AS ROW END NOT NULL,
    PERIOD FOR SYSTEM_TIME (SysStartTime,SysEndTime),
    CONSTRAINT [PK_TitleTypes] PRIMARY KEY CLUSTERED ([Id] ASC)

)
WITH (SYSTEM_VERSIONING = ON (HISTORY_TABLE = dbo.[TitleTypesHistory]));

我正在研究需要创建目录的全文搜索。如果我正确理解我的研究,这将极大地有益于需要在文本中搜索子字符串的人,而这里的情况并非如此。相反,我希望像字符串二进制索引这样的东西可以更快地确定我的搜索字符串是否包含在列中。

sql-server index-tuning
  • 2 个回答
  • 155 Views
Martin Hope
O. Jones
Asked: 2022-01-06 04:20:11 +0800 CST

顺序(自动递增)ID 是否需要唯一索引?

  • 2

如果顺序 AUTO_INCREMENT 列也不是 MySQL 表的主键,它是否必须有自己的 UNIQUE 索引才能确保长期数据完整性?

我正在重新索引一个特定的 MySQL 表以加速某些非常常见的 WHERE 子句。该表来自WordPress CMS。它的发行版外定义,简化,就是这样。

CREATE TABLE wp_postmeta (
    meta_id BIGINT UNSIGNED AUTO_INCREMENT,
    post_id BIGINT UNSIGNED,
    meta_key VARCHAR(255),
    meta_value LONGTEXT,
    PRIMARY KEY (meta_id),
    INDEX post_id (post_id),
    INDEX meta_key (meta_key)
);

MariaDB / MySQL (InnoDB) 现在使用聚集索引,所以我将 PK 更改为:

    PRIMARY KEY (post_id, meta_key, meta_id)

post_id = 42 AND meta_key = 'value'这意味着可以使用聚集索引和 LONGTEXT 获取来满足WHERE 过滤器等。事实证明,它可以解决我的用户的性能问题。

问题是:这个重新索引的表是否还需要在自动递增 ID 列上建立唯一索引才能长期保持正常工作?或者我可以省略该索引以节省一点空间和 UPDATE / INSERT 性能吗?

    UNIQUE INDEX meta_id (meta_id),

(我知道可能有比这个更好的表格设计。但是我目前的任务不允许我更改表格,只能更改索引。)

mysql index-tuning
  • 5 个回答
  • 435 Views
Martin Hope
Shane
Asked: 2021-08-28 00:10:26 +0800 CST

努力为我的 PSQL 查询创建索引

  • 0

我正在对包含 3000 万行并且只会继续增长的数据集运行查询,该表是 customer_actions(表大小为 2416 MB)

create table customer_actions
(
    id          bigint not null
        constraint customer_actions_pkey
            primary key,
    action      text,
    customer_id bigint,
    product_id  bigint,
    item_type   text,
    create_date timestamp
);

我尝试了各种各样的索引,但是查看查询的 exaplin,没有任何内容被击中

SELECT customer_id, product_id, count(*) AS count
from customer_actions
WHERE action = 'a2b'
      AND item_type = 'wine'
      AND create_date BETWEEN current_timestamp - INTERVAL '2 Years' AND current_timestamp
GROUP BY customer_id, product_id

SELECT customer_id, product_id, count(*) AS count
from customer_actions
WHERE action = 'view'
      AND item_type = 'wine'
      AND create_date BETWEEN current_timestamp - INTERVAL '2 Years' AND current_timestamp
GROUP BY customer_id, product_id

SELECT customer_id, product_id, count(*) AS count
from customer_actions
WHERE action = 'buy'
      AND item_type = 'wine'
      AND create_date BETWEEN current_timestamp - INTERVAL '2 Years' AND current_timestamp
GROUP BY customer_id, product_id

我尝试过的索引,其中一些我知道不会起作用但我抓住了稻草,所有最后有条件的索引也都尝试过没有条件。不要以为任何人都能为我指明正确的方向,我对 PSQL 还很陌生,对索引还没有深入的了解。

CREATE  INDEX IF NOT EXISTS idx_14 on customer_actions (customer_id, product_id, 
create_date, action, item_type) where action = 'a2b'
CREATE  INDEX IF NOT EXISTS idx_15 on customer_actions (customer_id, product_id, action, item_type) where action = 'a2b'
CREATE  INDEX IF NOT EXISTS idx_16 on customer_actions (customer_id, product_id) where action = 'a2b'
CREATE INDEX IF NOT EXISTS idx_11 on customer_actions (item_type, action ) where item_type = 'wine' and action = 'a2b';
CREATE INDEX IF NOT EXISTS idx_12 on customer_actions (item_type, action ) where item_type = 'wine' and action = 'view' ;
CREATE INDEX IF NOT EXISTS idx_13 on customer_actions (item_type, action ) where item_type = 'wine' and action = 'buy' ;
CREATE INDEX idx_time on customer_actions using brin (create_date);
create index idx_actions_a2b on customer_actions (action) where action = 'a2b'
CREATE INDEX IF NOT EXISTS idx_customer_actions_action_product_cardinality_order on customer_actions (customer_id, product_id, action);
CREATE INDEX id_time_and_other on customer_actions (action, item_type, create_date DESC)
CREATE INDEX IF NOT EXISTS idx_customer_actions_product_and_customer on customer_actions (customer_id, product_id)
CREATE  INDEX IF NOT EXISTS idx_14 on customer_actions (customer_id, product_id, create_date, action, item_type)
CREATE  INDEX IF NOT EXISTS idx_14 on customer_actions (customer_id, product_id, create_date, action)
CREATE  INDEX IF NOT EXISTS idx_17 on customer_actions (customer_id, product_id)

查询的解释是

Finalize GroupAggregate  (cost=745877.49..1182094.60 rows=1527687 width=24)
"  Group Key: customer_id, product_id"
  ->  Gather Merge  (cost=745877.49..1143902.43 rows=3055374 width=24)
        Workers Planned: 2
        ->  Partial GroupAggregate  (cost=744877.46..790236.43 rows=1527687 width=24)
"              Group Key: customer_id, product_id"
              ->  Sort  (cost=744877.46..752397.99 rows=3008210 width=16)
"                    Sort Key: customer_id, product_id"
                    ->  Parallel Seq Scan on customer_actions  (cost=0.00..318363.94 rows=3008210 width=16)
                          Filter: ((action = 'a2b'::text) AND (item_type = 'wine'::text) AND (create_date <= CURRENT_TIMESTAMP) AND (create_date >= (CURRENT_TIMESTAMP - '2 years'::interval)))

在此处输入图像描述

index index-tuning
  • 1 个回答
  • 23 Views
Martin Hope
Eliy Arlev
Asked: 2021-07-21 04:31:22 +0800 CST

聚集索引中列的顺序重要吗?

  • 2

我在索引中首先指定哪个字段是否重要?

将

CREATE SET TABLE my_table
(   
        validity_date DATE
    ,   branch_id SMALLINT 
    ,   account_class_id BYTEINT
    ,   catalog_id INTEGER
    ,   No_events FLOAT
)
PRIMARY INDEX (validity_date, branch_id);

被服务器视为相同

CREATE SET TABLE my_table
(   
        validity_date DATE
    ,   branch_id SMALLINT 
    ,   account_class_id BYTEINT
    ,   catalog_id INTEGER
    ,   No_events FLOAT
)
PRIMARY INDEX (branch_id, validity_date);
index-tuning clustered-index
  • 2 个回答
  • 112 Views
Martin Hope
Avi
Asked: 2021-07-10 07:08:43 +0800 CST

高效的维度和事实连接

  • -1

我有一个大的事实表和一个简单星型模式中的小得多的维度表:

--1.
CREATE TABLE dbo.Dim
(
Id INT NOT NULL IDENTITY PRIMARY KEY CLUSTERED,
CustomerName VARCHAR(2000)
)
--index
CREATE UNIQUE NONCLUSTERED INDEX uniqueindex1 ON Dim(CustomerName);


--2. 
CREATE TABLE dbo.Fact
(
...
PurchaseDate DATE 
CustomerNameId INT CONSTRAINT fk1 FOREIGN KEY (CustomerNameId) REFERENCES dbo.Dim(Id)
...
)
--index
CREATE CLUSTERED COLUMNSTORE INDEX ccs ON dbo.Fact;

运行以下简单查询,过滤事实表并加入维度:

SELECT sd.CustomerName,f.*
FROM dbo.Fact f
INNER JOIN dbo.Dim sd ON sd.Id = f.CustomerNameId
WHERE f.PurchaseDate IN (
'20000506',
'20000507',
'20000508',
'20000509',
'20000501',
'20000502',
'20000503'
)

我们得到以下丑陋的查询计划: 在此处输入图像描述

有趣的是,维度表倾向于在 4 次迭代中扫描所有 500 000 行,但最终在事实表的该日期范围内只需要几千行。

这对于较大的维度表来说是非常低效的,基本上所有的行都是一直扫描的,就像查找表索引甚至不存在一样。

预期的事情是sql server首先将事实表限制在日期范围内,然后使用这个有限的CustomerKeyId范围,它使用索引查找从小维度表中查找CustomerName。

  1. 这真的是星型模式的效率低下,还是我在这里想念什么?
  2. 换句话说,我怎么能强制 sql server 准备有限的 CustomerKeyId 表并只查找那些?(不知何故有CTE?)
index-tuning sql-server-2017
  • 1 个回答
  • 56 Views

Sidebar

Stats

  • 问题 205573
  • 回答 270741
  • 最佳答案 135370
  • 用户 68524
  • 热门
  • 回答
  • Marko Smith

    连接到 PostgreSQL 服务器:致命:主机没有 pg_hba.conf 条目

    • 12 个回答
  • Marko Smith

    如何让sqlplus的输出出现在一行中?

    • 3 个回答
  • Marko Smith

    选择具有最大日期或最晚日期的日期

    • 3 个回答
  • Marko Smith

    如何列出 PostgreSQL 中的所有模式?

    • 4 个回答
  • Marko Smith

    列出指定表的所有列

    • 5 个回答
  • Marko Smith

    如何在不修改我自己的 tnsnames.ora 的情况下使用 sqlplus 连接到位于另一台主机上的 Oracle 数据库

    • 4 个回答
  • Marko Smith

    你如何mysqldump特定的表?

    • 4 个回答
  • Marko Smith

    使用 psql 列出数据库权限

    • 10 个回答
  • Marko Smith

    如何从 PostgreSQL 中的选择查询中将值插入表中?

    • 4 个回答
  • Marko Smith

    如何使用 psql 列出所有数据库和表?

    • 7 个回答
  • Martin Hope
    Jin 连接到 PostgreSQL 服务器:致命:主机没有 pg_hba.conf 条目 2014-12-02 02:54:58 +0800 CST
  • Martin Hope
    Stéphane 如何列出 PostgreSQL 中的所有模式? 2013-04-16 11:19:16 +0800 CST
  • Martin Hope
    Mike Walsh 为什么事务日志不断增长或空间不足? 2012-12-05 18:11:22 +0800 CST
  • Martin Hope
    Stephane Rolland 列出指定表的所有列 2012-08-14 04:44:44 +0800 CST
  • Martin Hope
    haxney MySQL 能否合理地对数十亿行执行查询? 2012-07-03 11:36:13 +0800 CST
  • Martin Hope
    qazwsx 如何监控大型 .sql 文件的导入进度? 2012-05-03 08:54:41 +0800 CST
  • Martin Hope
    markdorison 你如何mysqldump特定的表? 2011-12-17 12:39:37 +0800 CST
  • Martin Hope
    Jonas 如何使用 psql 对 SQL 查询进行计时? 2011-06-04 02:22:54 +0800 CST
  • Martin Hope
    Jonas 如何从 PostgreSQL 中的选择查询中将值插入表中? 2011-05-28 00:33:05 +0800 CST
  • Martin Hope
    Jonas 如何使用 psql 列出所有数据库和表? 2011-02-18 00:45:49 +0800 CST

热门标签

sql-server mysql postgresql sql-server-2014 sql-server-2016 oracle sql-server-2008 database-design query-performance sql-server-2017

Explore

  • 主页
  • 问题
    • 最新
    • 热门
  • 标签
  • 帮助

Footer

AskOverflow.Dev

关于我们

  • 关于我们
  • 联系我们

Legal Stuff

  • Privacy Policy

Language

  • Pt
  • Server
  • Unix

© 2023 AskOverflow.DEV All Rights Reserve