我们正在运行本地 SQL Server 2017 来支持数据仓库数据库。数据库通过 SSIS 按计划加载,主要是通过使用临时表和 MERGE 功能。最近,我们开始看到错误“资源池'内部'中的系统内存不足,无法运行此查询。”。在过去的几周里,它变得越来越猖獗。
我们尝试过:
- 关闭查询存储
- 通过 CU27 更新 SQL Server
- 跟踪查询(没有好的结果)
- 运行尽可能多的报告以查明问题
与错误发生时间相关的最新更改:
- 打开查询存储
- 添加大量索引
其他信息:
- 我们在服务器上有 32gb 的内存并分配 26gb 给 sql server
- MERGE 的 TARGET 表是一个 CLUSTERED COLUMNSTORE INDEX
- MERGE 的 SOURCE 表是一个 HEAP
- 随着时间/多次失败,已暂存更改记录的 SOURCE 表已增长到超过 200,000 条记录。TARGET 表大约有 1000 万行。
任何帮助,将不胜感激。我在过去的几天里搜索了互联网,寻找任何指导。到目前为止,我所看到的是:
- 更新 SQL 版本
- 修改您的查询
- 向服务器添加内存
合并声明:
DROP TABLE IF EXISTS #Changes;
DROP TABLE IF EXISTS #TransformedChanges;
CREATE TABLE #Changes
(
[Change Type] VARCHAR(100)
);
MERGE [dbo].[FactOrderLine] AS TARGET
USING ( SELECT
[FactOrderLine].[OrderLine_Key],
[FactOrderLine].[BookedDate_Date_Key],
[FactOrderLine].[BookedDate_Time_Key],
[FactOrderLine].[Account_Key],
[FactOrderLine].[CCN_Key],
[FactOrderLine].[BillTo_SalesOffice_Key],
[FactOrderLine].[BillTo_Territory_Key],
[FactOrderLine].[ShipTo_SalesOffice_Key],
[FactOrderLine].[ShipTo_Territory_Key],
[FactOrderLine].[AssemblyLocation_Key],
[FactOrderLine].[ProductDivision_Key],
[FactOrderLine].[Product_Key],
[FactOrderLine].[Booked Date],
[FactOrderLine].[Ordered Quantity],
[FactOrderLine].[Unit Price - CCN],
[FactOrderLine].[Unit Price - Transaction],
[FactOrderLine].[Discount Factor],
[FactOrderLine].[Split Factor],
[FactOrderLine].[Is Split?],
[DW_Checksum] = CHECKSUM([FactOrderLine].[BookedDate_Date_Key],
[FactOrderLine].[BookedDate_Time_Key],
[FactOrderLine].[Account_Key],
[FactOrderLine].[CCN_Key],
[FactOrderLine].[BillTo_SalesOffice_Key],
[FactOrderLine].[BillTo_Territory_Key],
[FactOrderLine].[ShipTo_SalesOffice_Key],
[FactOrderLine].[ShipTo_Territory_Key],
[FactOrderLine].[AssemblyLocation_Key],
[FactOrderLine].[ProductDivision_Key],
[FactOrderLine].[Product_Key],
[FactOrderLine].[Booked Date],
[FactOrderLine].[Ordered Quantity],
[FactOrderLine].[Unit Price - CCN],
[FactOrderLine].[Unit Price - Transaction],
[FactOrderLine].[Discount Factor],
[FactOrderLine].[Split Factor],
[FactOrderLine].[Is Split?],
0)
FROM [changeLog].[FactOrderLine] ) AS SOURCE
ON [Source].[OrderLine_Key] = [Target].[OrderLine_Key]
WHEN MATCHED AND ISNULL([Source].[DW_Checksum], 0) <> ISNULL([Target].[DW_Checksum], 0) THEN UPDATE SET
[Target].[BookedDate_Date_Key] = [Source].[BookedDate_Date_Key],
[Target].[BookedDate_Time_Key] = [Source].[BookedDate_Time_Key],
[Target].[Account_Key] = [Source].[Account_Key],
[Target].[CCN_Key] = [Source].[CCN_Key],
[Target].[BillTo_SalesOffice_Key] = [Source].[BillTo_SalesOffice_Key],
[Target].[BillTo_Territory_Key] = [Source].[BillTo_Territory_Key],
[Target].[ShipTo_SalesOffice_Key] = [Source].[ShipTo_SalesOffice_Key],
[Target].[ShipTo_Territory_Key] = [Source].[ShipTo_Territory_Key],
[Target].[AssemblyLocation_Key] = [Source].[AssemblyLocation_Key],
[Target].[ProductDivision_Key] = [Source].[ProductDivision_Key],
[Target].[Product_Key] = [Source].[Product_Key],
[Target].[Booked Date] = [Source].[Booked Date],
[Target].[Ordered Quantity] = [Source].[Ordered Quantity],
[Target].[Unit Price - CCN] = [Source].[Unit Price - CCN],
[Target].[Unit Price - Transaction] = [Source].[Unit Price - Transaction],
[Target].[Discount Factor] = [Source].[Discount Factor],
[Target].[Split Factor] = [Source].[Split Factor],
[Target].[Is Split?] = [Source].[Is Split?],
[Target].[DW_Checksum] = [Source].[DW_Checksum],
[Target].[DW_ModifiedOn] = GETUTCDATE(),
[Target].[DW_IsDeleted?] = 0
WHEN NOT MATCHED BY TARGET THEN INSERT
(
[OrderLine_Key],
[BookedDate_Date_Key],
[BookedDate_Time_Key],
[Account_Key],
[CCN_Key],
[BillTo_SalesOffice_Key],
[BillTo_Territory_Key],
[ShipTo_SalesOffice_Key],
[ShipTo_Territory_Key],
[AssemblyLocation_Key],
[ProductDivision_Key],
[Product_Key],
[Booked Date],
[Ordered Quantity],
[Unit Price - CCN],
[Unit Price - Transaction],
[Discount Factor],
[Split Factor],
[Is Split?],
[DW_IsDeleted?], [DW_Checksum], [Source_ModifiedOn], [DW_ModifiedOn], [DW_CreatedOn] ) VALUES (
[Source].[OrderLine_Key],
[Source].[BookedDate_Date_Key],
[Source].[BookedDate_Time_Key],
[Source].[Account_Key],
[Source].[CCN_Key],
[Source].[BillTo_SalesOffice_Key],
[Source].[BillTo_Territory_Key],
[Source].[ShipTo_SalesOffice_Key],
[Source].[ShipTo_Territory_Key],
[Source].[AssemblyLocation_Key],
[Source].[ProductDivision_Key],
[Source].[Product_Key],
[Source].[Booked Date],
[Source].[Ordered Quantity],
[Source].[Unit Price - CCN],
[Source].[Unit Price - Transaction],
[Source].[Discount Factor],
[Source].[Split Factor],
[Source].[Is Split?],
0,
[Source].[DW_Checksum],NULL,GETUTCDATE(),GETUTCDATE()
)
OUTPUT $action INTO #Changes;
CREATE TABLE #TransformedChanges
(
[Update Record Count] INT,
[Insert Record Count] INT
);
INSERT INTO #TransformedChanges
SELECT *
FROM
(
SELECT
TRIM(#Changes.[Change Type])+' Record Count' AS [Change Type],
COUNT(*) AS [Record Count]
FROM #Changes
GROUP BY TRIM(#Changes.[Change Type])+' Record Count'
) A
PIVOT
(
SUM([Record Count])
FOR A.[Change Type] IN ([Update Record Count], [Insert Record Count])
) [B];
DECLARE @TransformationChecker INT;
SET @TransformationChecker = (SELECT COUNT(*) FROM #TransformedChanges);
IF @TransformationChecker = 0 INSERT INTO #TransformedChanges VALUES(0,0);
SELECT ISNULL([Update Record Count], 0) [Update Record Count], ISNULL([Insert Record Count], 0) [Insert Record Count] FROM #TransformedChanges;
DROP TABLE IF EXISTS #Changes;
DROP TABLE IF EXISTS #TransformedChanges;
感谢多位评论者的帮助,我发现我的问题已通过删除 MERGE 语句的使用得到解决,而是运行独立的 INSERT/UPDATE/DELETE 语句。这是由多个博客、与我一起工作的 DBA 和这里的评论者推荐的,但由于没有可用的文档表明 MERGE 和内存问题可能相关(以及我自己的固执),我没有尝试切换它们,直到尝试了多个其他路线。
我仍然很好奇究竟是什么导致了内存问题,而表面上你会期望相同数量的更新、插入和删除需要相同(或相似)数量的资源。显然,MERGE 语句不是以这种方式运行的,有时您必须对解决方案进行实际操作。
“资源池‘内部’中的系统内存不足,无法运行此查询。” 检查与错误日志中的错误消息关联的 spid。因为合并的目标是聚集列存储索引,所以我强烈怀疑该错误与 MERGE 没有直接关联,而是 MERGE 的结果。来自 MERGE 本身的消息不应该在内部资源池中。
后台元组移动器确实在内部资源池中运行。除非使用 (COMPRESS_ALL_ROW_GROUPS = ON) 的重组压缩任何剩余的增量存储,否则元组移动器将在某些时候关闭并压缩增量存储。它需要内存授权才能这样做。如果元组移动器的内存授权超时或元组移动器的最小授权不可用,则结果是该错误。我相信元组移动器总是使用 1 GB 的内存授权。