AskOverflow.Dev

AskOverflow.Dev Logo AskOverflow.Dev Logo

AskOverflow.Dev Navigation

  • Início
  • system&network
  • Ubuntu
  • Unix
  • DBA
  • Computer
  • Coding
  • LangChain

Mobile menu

Close
  • Início
  • system&network
    • Recentes
    • Highest score
    • tags
  • Ubuntu
    • Recentes
    • Highest score
    • tags
  • Unix
    • Recentes
    • tags
  • DBA
    • Recentes
    • tags
  • Computer
    • Recentes
    • tags
  • Coding
    • Recentes
    • tags
Início / dba / Perguntas / 116552
Accepted
mpag
mpag
Asked: 2015-09-30 20:14:36 +0800 CST2015-09-30 20:14:36 +0800 CST 2015-09-30 20:14:36 +0800 CST

Access (Jet) SQL: carimbos de data e hora na TabelaB flanqueando cada carimbo de data e hora na TabelaA

  • 772

Primeiras palavras

Você pode ignorar com segurança as seções abaixo (e incluindo) JOINs: Starting Off se você quiser apenas decifrar o código. O pano de fundo e os resultados servem apenas como contexto. Por favor, olhe o histórico de edições antes de 2015-10-06 se você quiser ver como era o código inicialmente.


Objetivo

Por fim, quero calcular as coordenadas GPS interpoladas para o transmissor ( Xou Xmit) com base nos carimbos DateTime dos dados GPS disponíveis na tabela SecondTableque flanqueiam diretamente a observação na tabela FirstTable.

Meu objetivo imediato para atingir o objetivo final é descobrir a melhor forma de se juntar FirstTablepara SecondTableobter esses pontos de tempo de flanco. Mais tarde, posso usar essas informações e calcular coordenadas GPS intermediárias assumindo um ajuste linear ao longo de um sistema de coordenadas equiretangulares (palavras elegantes para dizer que não me importo que a Terra seja uma esfera nessa escala).


Perguntas

  1. Existe uma maneira mais eficiente de gerar os carimbos de data/hora antes e depois mais próximos?
    • Corrigido por mim mesmo, apenas pegando o "depois" e, em seguida, obtendo o "antes" apenas conforme relacionado ao "depois".
  2. Existe uma maneira mais intuitiva que não envolva a (A<>B OR A=B)estrutura.
    • Byrdzeye forneceu as alternativas básicas, no entanto, minha experiência no "mundo real" não se alinhou com todas as 4 de suas estratégias de junção com o mesmo desempenho. Mas todo o crédito a ele por abordar os estilos alternativos de junção.
  3. Quaisquer outros pensamentos, truques e conselhos que você possa ter.
    • Até agora, tanto byrdzeye quanto Phrancis foram bastante úteis a esse respeito. Achei que o conselho de Phrancis foi excelentemente apresentado e forneceu ajuda em um estágio crítico, então vou dar a ele uma vantagem aqui.

Eu ainda gostaria de qualquer ajuda adicional que eu possa receber em relação à pergunta 3. Os marcadores refletem quem eu acredito que mais me ajudou na pergunta individual.


Definições de tabela

Representação semivisual

PrimeiraTabela

Fields
  RecTStamp | DateTime  --can contain milliseconds via VBA code (see Ref 1) 
  ReceivID  | LONG
  XmitID    | TEXT(25)
Keys and Indices
  PK_DT     | Primary, Unique, No Null, Compound
    XmitID    | ASC
    RecTStamp | ASC
    ReceivID  | ASC
  UK_DRX    | Unique, No Null, Compound
    RecTStamp | ASC
    ReceivID  | ASC
    XmitID    | ASC

SegundaTabela

Fields
  X_ID      | LONG AUTONUMBER -- seeded after main table has been created and already sorted on the primary key
  XTStamp   | DateTime --will not contain partial seconds
  Latitude  | Double   --these are in decimal degrees, not degrees/minutes/seconds
  Longitude | Double   --this way straight decimal math can be performed
Keys and Indices
  PK_D      | Primary, Unique, No Null, Simple
    XTStamp   | ASC
  UIDX_ID   | Unique, No Null, Simple
    X_ID      | ASC

Tabela de detalhes do receptor

Fields
  ReceivID                      | LONG
  Receiver_Location_Description | TEXT -- NULL OK
  Beginning                     | DateTime --no partial seconds
  Ending                        | DateTime --no partial seconds
  Lat                           | DOUBLE
  Lon                           | DOUBLE
Keys and Indicies
  PK_RID  | Primary, Unique, No Null, Simple
    ReceivID | ASC

tabela ValidXmitters

Field (and primary key)
  XmitID    | TEXT(25) -- primary, unique, no null, simple

violino SQL...

...para que você possa brincar com as definições e o código da tabela Esta pergunta é para o MSAccess, mas, como Phrancis apontou, não há um estilo SQL fiddle para o Access. Portanto, você deve poder acessar aqui para ver minhas definições de tabela e código com base na resposta de Phrancis :
http://sqlfiddle.com/#!6/e9942/4 (link externo)


JOINs: Começando

Minha estratégia JOIN atual de "entranhas internas"

Primeiro, crie um FirstTable_rekeyed com a ordem das colunas e a chave primária composta, (RecTStamp, ReceivID, XmitID)todas indexadas/classificadas ASC. Também criei índices em cada coluna individualmente. Em seguida, preencha-o assim.

INSERT INTO FirstTable_rekeyed (RecTStamp, ReceivID, XmitID)
  SELECT DISTINCT ROW RecTStamp, ReceivID, XmitID
  FROM FirstTable
  WHERE XmitID IN (SELECT XmitID from ValidXmitters)
  ORDER BY RecTStamp, ReceivID, XmitID;

A consulta acima preenche a nova tabela com 153.006 registros e retorna em cerca de 10 segundos.

O seguinte é concluído em um ou dois segundos quando todo esse método é agrupado em um "SELECT Count(*) FROM ( ... )" quando o método de subconsulta TOP 1 é usado

SELECT 
    ReceiverRecord.RecTStamp, 
    ReceiverRecord.ReceivID, 
    ReceiverRecord.XmitID,
    (SELECT TOP 1 XmitGPS.X_ID FROM SecondTable as XmitGPS WHERE ReceiverRecord.RecTStamp < XmitGPS.XTStamp ORDER BY XmitGPS.X_ID) AS AfterXmit_ID
    FROM FirstTable_rekeyed AS ReceiverRecord
    -- INNER JOIN SecondTable AS XmitGPS ON (ReceiverRecord.RecTStamp < XmitGPS.XTStamp)
         GROUP BY RecTStamp, ReceivID, XmitID;
-- No separate join needed for the Top 1 method, but it would be required for the other methods. 
-- Additionally no restriction of the returned set is needed if I create the _rekeyed table.
-- May not need GROUP BY either. Could try ORDER BY.
-- The three AfterXmit_ID alternatives below take longer than 3 minutes to complete (or do not ever complete).
  -- FIRST(XmitGPS.X_ID)
  -- MIN(XmitGPS.X_ID)
  -- MIN(SWITCH(XmitGPS.XTStamp > ReceiverRecord.RecTStamp, XmitGPS.X_ID, Null))

Consulta JOIN anterior "inner guts"

Primeiro (rápido... mas não bom o suficiente)

SELECT 
  A.RecTStamp,
  A.ReceivID,
  A.XmitID,
  MAX(IIF(B.XTStamp<= A.RecTStamp,B.XTStamp,Null)) as BeforeXTStamp,
  MIN(IIF(B.XTStamp > A.RecTStamp,B.XTStamp,Null)) as AfterXTStamp
FROM FirstTable as A
INNER JOIN SecondTable as B ON 
  (A.RecTStamp<>B.XTStamp OR A.RecTStamp=B.XTStamp)
GROUP BY A.RecTStamp, A.ReceivID, A.XmitID
  -- alternative for BeforeXTStamp MAX(-(B.XTStamp<=A.RecTStamp)*B.XTStamp)
  -- alternatives for AfterXTStamp (see "Aside" note below)
  -- 1.0/(MAX(1.0/(-(B.XTStamp>A.RecTStamp)*B.XTStamp)))
  -- -1.0/(MIN(1.0/((B.XTStamp>A.RecTStamp)*B.XTStamp)))

Segundo (mais lento)

SELECT
  A.RecTStamp, AbyB1.XTStamp AS BeforeXTStamp, AbyB2.XTStamp AS AfterXTStamp
FROM (FirstTable AS A INNER JOIN 
  (select top 1 B1.XTStamp, A1.RecTStamp 
   from SecondTable as B1, FirstTable as A1
   where B1.XTStamp<=A1.RecTStamp
   order by B1.XTStamp DESC) AS AbyB1 --MAX (time points before)
ON A.RecTStamp = AbyB1.RecTStamp) INNER JOIN 
  (select top 1 B2.XTStamp, A2.RecTStamp 
   from SecondTable as B2, FirstTable as A2
   where B2.XTStamp>A2.RecTStamp
   order by B2.XTStamp ASC) AS AbyB2 --MIN (time points after)
ON A.RecTStamp = AbyB2.RecTStamp; 

Fundo

Eu tenho uma tabela de telemetria (com o apelido de A) de pouco menos de 1 milhão de entradas com uma chave primária composta baseada em um DateTimecarimbo, uma ID de transmissor e uma ID de dispositivo de gravação. Devido a circunstâncias fora do meu controle, minha linguagem SQL é o Jet DB padrão no Microsoft Access (os usuários usarão 2007 e versões posteriores). Apenas cerca de 200.000 dessas entradas são relevantes para a consulta devido ao ID do transmissor.

Existe uma segunda tabela de telemetria (alias B) que envolve aproximadamente 50.000 entradas com uma única DateTimechave primária

Na primeira etapa, concentrei-me em encontrar os carimbos de data/hora mais próximos dos carimbos da primeira tabela a partir da segunda tabela.


Resultados do JOIN

Curiosidades que descobri...

...ao longo do caminho durante a depuração

É realmente estranho escrever a JOINlógica como FROM FirstTable as A INNER JOIN SecondTable as B ON (A.RecTStamp<>B.XTStamp OR A.RecTStamp=B.XTStamp)@byrdzeye apontou em um comentário (que desde então desapareceu) é uma forma de junção cruzada. Observe que substituir LEFT OUTER JOINno INNER JOINcódigo acima parece não causar impacto na quantidade ou identidade das linhas retornadas. Também não consigo deixar de fora a cláusula ON ou dizer ON (1=1). Apenas usar uma vírgula para unir (em vez de INNERou LEFT OUTER JOIN) resulta em Count(select * from A) * Count(select * from B)linhas retornadas nesta consulta, em vez de apenas uma linha por tabela A, como o JOINretorno explícito (A<>B OR A=B). Isso claramente não é adequado. FIRSTnão parece estar disponível para uso dado um tipo de chave primária composta.

O segundo JOINestilo, embora indiscutivelmente mais legível, sofre por ser mais lento. Isso pode ocorrer porque dois JOINs internos adicionais são necessários contra a tabela maior, bem como os dois CROSS JOINs encontrados em ambas as opções.

Aparte: Substituir a IIFcláusula por MIN/ MAXparece retornar o mesmo número de entradas.
MAX(-(B.XTStamp<=A.RecTStamp)*B.XTStamp)
funciona para o timestamp "Antes" ( MAX), mas não funciona diretamente para o "Depois" ( MIN) da seguinte forma:
MIN(-(B.XTStamp>A.RecTStamp)*B.XTStamp)
porque o mínimo é sempre 0 para a FALSEcondição. Esse 0 é menor do que qualquer pós-época DOUBLE(do qual um DateTimecampo é um subconjunto no Access e no qual esse cálculo transforma o campo). Os métodos IIFe MIN/ MAXAs alternativas propostas para o valor AfterXTStamp funcionam porque a divisão por zero ( FALSE) gera valores nulos, que as funções de agregação MIN e MAX ignoram.

Próximos passos

Levando isso adiante, desejo encontrar os timestamps na segunda tabela que flanqueiam diretamente os timestamps na primeira tabela e realizar uma interpolação linear dos valores de dados da segunda tabela com base na distância de tempo para esses pontos (ou seja, se o timestamp de a primeira tabela está a 25% do caminho entre o "antes" e o "depois", gostaria que 25% do valor calculado viesse dos dados do valor da 2ª tabela associados ao ponto "depois" e 75% do "antes" ). Usando o tipo de junção revisado como parte das entranhas internas, e após as respostas sugeridas abaixo, eu produzo ...

    SELECT
        AvgGPS.XmitID,
        StrDateIso8601Msec(AvgGPS.RecTStamp) AS RecTStamp_ms,
        -- StrDateIso8601MSec is a VBA function returning a TEXT string in yyyy-mm-dd hh:nn:ss.lll format
        AvgGPS.ReceivID,
        RD.Receiver_Location_Description,
        RD.Lat AS Receiver_Lat,
        RD.Lon AS Receiver_Lon,
        AvgGPS.Before_Lat * (1 - AvgGPS.AfterWeight) + AvgGPS.After_Lat * AvgGPS.AfterWeight AS Xmit_Lat,
        AvgGPS.Before_Lon * (1 - AvgGPS.AfterWeight) + AvgGPS.After_Lon * AvgGPS.AfterWeight AS Xmit_Lon,
        AvgGPS.RecTStamp AS RecTStamp_basic
    FROM ( SELECT 
        AfterTimestampID.RecTStamp,
        AfterTimestampID.XmitID,
        AfterTimestampID.ReceivID,
        GPSBefore.BeforeXTStamp, 
        GPSBefore.Latitude AS Before_Lat, 
        GPSBefore.Longitude AS Before_Lon,
        GPSAfter.AfterXTStamp, 
        GPSAfter.Latitude AS After_Lat, 
        GPSAfter.Longitude AS After_Lon,
        ( (AfterTimestampID.RecTStamp - GPSBefore.XTStamp) / (GPSAfter.XTStamp - GPSBefore.XTStamp) ) AS AfterWeight
        FROM (
            (SELECT 
                ReceiverRecord.RecTStamp, 
                ReceiverRecord.ReceivID, 
                ReceiverRecord.XmitID,
               (SELECT TOP 1 XmitGPS.X_ID FROM SecondTable as XmitGPS WHERE ReceiverRecord.RecTStamp < XmitGPS.XTStamp ORDER BY XmitGPS.X_ID) AS AfterXmit_ID
             FROM FirstTable AS ReceiverRecord 
             -- WHERE ReceiverRecord.XmitID IN (select XmitID from ValidXmitters)
             GROUP BY RecTStamp, ReceivID, XmitID
            ) AS AfterTimestampID INNER JOIN SecondTable AS GPSAfter ON AfterTimestampID.AfterXmit_ID = GPSAfter.X_ID
        ) INNER JOIN SecondTable AS GPSBefore ON AfterTimestampID.AfterXmit_ID = GPSBefore.X_ID + 1
    ) AS AvgGPS INNER JOIN ReceiverDetails AS RD ON (AvgGPS.ReceivID = RD.ReceivID) AND (AvgGPS.RecTStamp BETWEEN RD.Beginning AND RD.Ending)
    ORDER BY AvgGPS.RecTStamp, AvgGPS.ReceivID;

...que retorna 152928 registros, correspondendo (pelo menos aproximadamente) ao número final de registros esperados. O tempo de execução é provavelmente de 5 a 10 minutos no meu sistema i7-4790, 16 GB de RAM, sem SSD, Win 8.1 Pro.


Referência 1: o MS Access pode lidar com valores de tempo de milissegundos - realmente e arquivo de origem [08080011.txt]

join ms-access
  • 3 3 respostas
  • 685 Views

3 respostas

  • Voted
  1. Best Answer
    Phrancis
    2015-10-03T21:21:58+08:002015-10-03T21:21:58+08:00

    I must first compliment you on your courage to do something like this with an Access DB, which from my experience is very difficult to do anything SQL-like. Anyways, on to the review.


    First join

    Your IIF field selections might benefit from using a Switch statement instead. It seems to be sometimes the case, especially with things SQL, that a SWITCH (more commonly known as CASE in typical SQL) is quite fast when just making simple comparisons in the body of a SELECT. The syntax in your case would be almost identical, although a switch can be expanded to cover a large chunk of comparisons in one field. Something to consider.

      SWITCH (
        expr1, val1,
        expr2, val2,
        val3        -- default value or "else"
      )
    

    A switch can also help readability, in larger statements. In context:

      MAX(SWITCH(B.XTStamp <= A.RecTStamp,B.XTStamp,Null)) as BeforeXTStamp,
      --alternatively MAX(-(B.XTStamp<=A.RecTStamp)*B.XTStamp) as BeforeXTStamp,
      MIN(SWITCH(B.XTStamp>A.RecTStamp,B.XTStamp,Null)) as AfterXTStamp
    

    As for the join itself, I think (A.RecTStamp<>B.XTStamp OR A.RecTStamp=B.XTStamp) is about as good as you're going to get, given what you are trying to do. It's not that fast, but I wouldn't expect it to be either.


    Second join

    You said this is slower. It's also less readable from a code standpoint. Given equally satisfactory result sets between 1 and 2, I'd say go for 1. At least it's obvious what you are trying to do that way. Subqueries are often not very fast (though often unavoidable) especially in this case you are throwing in an extra join in each, which must certainly complicate the execution plan.

    One remark, I saw that you used old ANSI-89 join syntax. It's best to avoid that, the performance will be same or better with the more modern join syntax, and they are less ambiguous or easier to read, harder to make mistakes.

    FROM (FirstTable AS A INNER JOIN 
      (select top 1 B1.XTStamp, A1.RecTStamp 
       from SecondTable as B1
       inner join FirstTable as A1
         on B1.XTStamp <= A1.RecTStamp
       order by B1.XTStamp DESC) AS AbyB1 --MAX (time points before)
    

    Naming things

    I think the way your things are named is unhelpful at best, and cryptic at worst. A, B, A1, B1 etc. as table aliases I think could be better. Also, I think the field names are not very good, but I realize you may not have control over this. I will just quickly quote The Codeless Code on the topic of naming things, and leave it at that...

    “Invective!” answered the priestess. “Verb your expletive nouns!”


    "Next steps" query

    I couldn't make much sense of it how it was written, I had to take it to a text editor and do some style changes to make it more readable. I know Access' SQL editor is beyond clunky, so I usually write my queries in a good editor like Notepad++ or Sublime Text. Some of the stylistic changes I applied to make it more readable:

    • 4 spaces indent instead of 2 spaces
    • Spaces around mathematical and comparison operators
    • More natural placing of braces and indentation (I went with Java-style braces, but could also be C-style, at your preference)

    So as it turns out, this is a very complicated query indeed. To make sense of it, I have to start from the innermost query, your ID data set, which I understand is the same as your First Join. It returns the IDs and timestamps of the devices where the before/after timestamps are the closest, within the subset of devices you are interested in. So instead of ID why not call it ClosestTimestampID.

    Your Det join is used only once:

    enter image description here

    The rest of the time, it only joins the values you already have from ClosestTimestampID. So instead we should be able to just do this:

        ) AS ClosestTimestampID
        INNER JOIN SecondTable AS TL1 
            ON ClosestTimestampID.BeforeXTStamp = TL1.XTStamp) 
        INNER JOIN SecondTable AS TL2 
            ON ClosestTimestampID.AfterXTStamp = TL2.XTStamp
        WHERE ClosestTimestampID.XmitID IN (<limited subset S>)
    

    Maybe not be a huge performance gain, but anything we can do to help the poor Jet DB optimizer will help!


    I can't shake the feeling that the calculations/algorithm for BeforeWeight and AfterWeight which you use to interpolate could be done better, but unfortunately I'm not very good with those.

    One suggestion to avoid crashing (although it's not ideal depending on your application) would be to break out your nested subqueries into tables of their own and update those when needed. I'm not sure how often you need your source data to be refreshed, but if it is not too often you might think of writing some VBA code to schedule an update of the tables and derived tables, and just leave your outermost query to pull from those tables instead of the original source. Just a thought, like I said not ideal but given the tool you may not have a choice.


    Everything together:

    SELECT
        InGPS.XmitID,
        StrDateIso8601Msec(InGPS.RecTStamp) AS RecTStamp_ms,
           -- StrDateIso8601MSec is a VBA function returning a TEXT string in yyyy-mm-dd hh:nn:ss.lll format
        InGPS.ReceivID,
        RD.Receiver_Location_Description,
        RD.Lat AS Receiver_Lat,
        RD.Lon AS Receiver_Lon,
        InGPS.Before_Lat * InGPS.BeforeWeight + InGPS.After_Lat * InGPS.AfterWeight AS Xmit_Lat,
        InGPS.Before_Lon * InGPS.BeforeWeight + InGPS.After_Lon * InGPS.AfterWeight AS Xmit_Lon,
        InGPS.RecTStamp AS RecTStamp_basic
    FROM (
        SELECT 
            ClosestTimestampID.RecTStamp,
            ClosestTimestampID.XmitID,
            ClosestTimestampID.ReceivID,
            ClosestTimestampID.BeforeXTStamp, 
            TL1.Latitude AS Before_Lat, 
            TL1.Longitude AS Before_Lon,
            (1 - ((ClosestTimestampID.RecTStamp - ClosestTimestampID.BeforeXTStamp) 
                / (ClosestTimestampID.AfterXTStamp - ClosestTimestampID.BeforeXTStamp))) AS BeforeWeight,
            ClosestTimestampID.AfterXTStamp, 
            TL2.Latitude AS After_Lat, 
            TL2.Longitude AS After_Lon,
            (     (ClosestTimestampID.RecTStamp - ClosestTimestampID.BeforeXTStamp) 
                / (ClosestTimestampID.AfterXTStamp - ClosestTimestampID.BeforeXTStamp)) AS AfterWeight
            FROM (((
                SELECT 
                    A.RecTStamp, 
                    A.ReceivID, 
                    A.XmitID,
                    MAX(SWITCH(B.XTStamp <= A.RecTStamp, B.XTStamp, Null)) AS BeforeXTStamp,
                    MIN(SWITCH(B.XTStamp > A.RecTStamp, B.XTStamp, Null)) AS AfterXTStamp
                FROM FirstTable AS A
                INNER JOIN SecondTable AS B 
                    ON (A.RecTStamp <> B.XTStamp OR A.RecTStamp = B.XTStamp)
                WHERE A.XmitID IN (<limited subset S>)
                GROUP BY A.RecTStamp, ReceivID, XmitID
            ) AS ClosestTimestampID
            INNER JOIN FirstTable AS Det 
                ON (Det.XmitID = ClosestTimestampID.XmitID) 
                AND (Det.ReceivID = ClosestTimestampID.ReceivID) 
                AND (Det.RecTStamp = ClosestTimestampID.RecTStamp)) 
            INNER JOIN SecondTable AS TL1 
                ON ClosestTimestampID.BeforeXTStamp = TL1.XTStamp) 
            INNER JOIN SecondTable AS TL2 
                ON ClosestTimestampID.AfterXTStamp = TL2.XTStamp
            WHERE Det.XmitID IN (<limited subset S>)
        ) AS InGPS
    INNER JOIN ReceiverDetails AS RD 
        ON (InGPS.ReceivID = RD.ReceivID) 
        AND (InGPS.RecTStamp BETWEEN <valid parameters from another table>)
    ORDER BY StrDateIso8601Msec(InGPS.RecTStamp), InGPS.ReceivID;
    
    • 10
  2. byrdzeye
    2015-10-02T06:50:08+08:002015-10-02T06:50:08+08:00
    • Adicionado atributos adicionais e condições de filtro.
    • Qualquer forma de junção cruzada é eliminada usando consultas aninhadas mínimas e máximas. Este é o maior ganho de desempenho.
    • Os valores de flanco mínimo e máximo retornados pela consulta mais aninhada interna são valores de chave primária (varreduras) que são usados ​​para recuperar atributos de flanco adicionais (lat e lon) usando uma busca por cálculos finais (o acesso tem um equivalente de aplicação).
    • Os atributos primários das tabelas são recuperados e filtrados na consulta mais interna e devem ajudar no desempenho.
    • Não há necessidade de formatar (StrDateIso8601Msec) o valor de hora para classificação. Usar o valor de data e hora da tabela é equivalente.

    Planos de execução do SQL Server (porque o Access não pode mostrar isso)
    Sem a ordem final porque é caro:
    Clustered Index Scan [ReceiverDetails].[PK_ReceiverDetails] Custo 16%
    Clustered Index Seek [FirstTable].[PK_FirstTable] Custo 19%
    Clustered Index Seek [SecondTable].[PK_SecondTable] Custo 16%
    Clustered Index Seek [SecondTable].[PK_SecondTable] Custo 16%
    Clustered Index Seek [SecondTable].[PK_SecondTable] [TL2] Custo 16%
    Clustered Index Seek [SecondTable].[PK_SecondTable] [TL1] Custo 16%

    Com a ordem final por:
    Ordenar Custo 36%
    Varredura de Índice Clusterizado [ReceiverDetails].[PK_ReceiverDetails] Custo 10% Busca de
    Índice Clusterizado [FirstTable].[PK_FirstTable] Custo 12%
    Busca de índice clusterizado [SecondTable].[PK_SecondTable] Custo 10% Busca de
    índice clusterizado [SecondTable].[PK_SecondTable] Custo Busca de índice clusterizado de 10% [SecondTable].[PK_SecondTable] [TL2] Custo Busca de índice agrupado
    de 10%
    [SecondTable].[ PK_SecondTable] [TL1] Custo 10%

    Código:

    select
         ClosestTimestampID.XmitID
        --,StrDateIso8601Msec(InGPS.RecTStamp) AS RecTStamp_ms
        ,ClosestTimestampID.ReceivID
        ,ClosestTimestampID.Receiver_Location_Description
        ,ClosestTimestampID.Lat
        ,ClosestTimestampID.Lon
    ,[TL1].[Latitude] * (1 - ((ClosestTimestampID.RecTStamp - ClosestTimestampID.BeforeXTStamp) / (ClosestTimestampID.AfterXTStamp - ClosestTimestampID.BeforeXTStamp))) + [TL2].[Latitude] * ((ClosestTimestampID.RecTStamp - ClosestTimestampID.BeforeXTStamp) / (ClosestTimestampID.AfterXTStamp - ClosestTimestampID.BeforeXTStamp)) AS Xmit_Lat
    ,[TL1].[Longitude] * (1 - ((ClosestTimestampID.RecTStamp - ClosestTimestampID.BeforeXTStamp) / (ClosestTimestampID.AfterXTStamp - ClosestTimestampID.BeforeXTStamp))) + [TL2].[Longitude] * ((ClosestTimestampID.RecTStamp - ClosestTimestampID.BeforeXTStamp) / (ClosestTimestampID.AfterXTStamp - ClosestTimestampID.BeforeXTStamp)) AS Xmit_Lon
        ,ClosestTimestampID.RecTStamp as RecTStamp_basic
    from (
            (
                (
                    select
                         FirstTable.RecTStamp
                        ,FirstTable.ReceivID
                        ,FirstTable.XmitID
                        ,ReceiverDetails.Receiver_Location_Description
                        ,ReceiverDetails.Lat
                        ,ReceiverDetails.Lon
                        ,(
                            select max(XTStamp) as val
                            from SecondTable
                            where XTStamp <= FirstTable.RecTStamp
                         ) as BeforeXTStamp
                        ,(
                            select min(XTStamp) as val
                            from SecondTable
                            where XTStamp > FirstTable.RecTStamp
                         ) as AfterXTStamp
                    from FirstTable
                    inner join ReceiverDetails
                    on ReceiverDetails.ReceivID = FirstTable.ReceivID
                    where FirstTable.RecTStamp between #1/1/1990# and #1/1/2020#
                    and FirstTable.XmitID in (100,110)
                ) as ClosestTimestampID
                inner join SecondTable as TL1
                on ClosestTimestampID.BeforeXTStamp = TL1.XTStamp
            )
            inner join SecondTable as TL2
            on ClosestTimestampID.AfterXTStamp = TL2.XTStamp
        )
    order by ClosestTimestampID.RecTStamp, ClosestTimestampID.ReceivID;
    

    Teste de desempenho da minha consulta em relação à consulta que contém a junção cruzada.

    FirstTable was loaded with 13 records and SecondTable with 1,000,000.
    The execution plans for my query didn't change much from what has been posted.
    Execution plans for the cross join:
    Nested Loops Cost 81% using INNER JOIN SecondTable AS B ON (A.RecTStamp <> B.XTStamp OR A.RecTStamp = B.XTStamp
    Nested Loops drops to 75% if using CROSS JOIN SecondTable AS B' or ',SecondTable AS B
    Stream Aggregate 8%
    Index Scan [SecondTable][UK_ID][B] 6%
    Table Spool 5%
    Several other Clustered Index Seek and Index Seeks (similar to my query as posted) with Cost of 0%.

    Execution time is .007 and 8-9 seconds for my query and the CROSS JOIN.
    Cost comparison 0% and 100%.

    I loaded FirstTable with 50,000 records and a single record to ReceiverDetails for a join condition and ran my query.
    50,013 returned between 0.9 and 1.0 second.

    I ran second query with the cross join and allowed it to run for about 20 minutes before I killed it.
    If the cross join query is filtered to return only the original 13, execution time is again, 8-9 seconds.
    Placement of the filter condition was at inner most select, outer most select and both. No difference.

    There is a difference between these two join conditions in favor of the CROSS JOIN, the first uses a predicate, the CROSS JOIN does not:
    INNER JOIN SecondTable AS B ON (A.RecTStamp <> B.XTStamp OR A.RecTStamp = B.XTStamp) CROSS JOIN SecondTable AS B

    • 5
  3. byrdzeye
    2015-10-18T07:27:46+08:002015-10-18T07:27:46+08:00

    Adding a second answer, not better than the first but without changing any of the requirements presented, there are a few of ways to beat Access into submission and appear snappy. 'Materialize' the complications a bit at a time effectivity using 'triggers'. Access tables do not have triggers so intercept and inject the crud processes.

    --*** Create a table for flank values.
        create table Flank (
             RecTStamp      datetime not null
            ,BeforeXTStamp  datetime null
            ,AfterXTStamp   datetime null
            ,constraint PK_Flank primary key clustered ( RecTStamp asc )
            )
    
    --*** Create a FlankUpdateLoop sub. (create what is missing)
        -- loop until rowcount < 5000 or rowcount = 0
        -- a 5K limit appears to be manageable for Access, especially for the initial population.
        insert into Flank (
             RecTStamp
            ,BeforeXTStamp
            ,AfterXTStamp
            )
        select top 5000 FirstTable.RecTStamp
            ,(
                select max(XTStamp) as val
                from SecondTable
                where XTStamp <= FirstTable.RecTStamp
                ) as BeforeXTStamp
            ,(
                select min(XTStamp) as val
                from SecondTable
                where XTStamp > FirstTable.RecTStamp
                ) as AfterXTStamp
        from FirstTable
        left join Flank
            on FirstTable.RecTStamp = Flank.RecTStamp
        where Flank.RecTStamp is null;
    
    --*** For FirstTable Adds, Changes or Deletes:
        delete from Flank where Flank.RecTStamp = CRUD_RecTStamp
        execute FlankUpdateLoop --See above. This will handle Adds, Changes or Deletes.
    
    --*** For SecondTable Adds, Changes or Deletes:
        --delete from Flank where the old value is immediately before and after the new flank value.
        --They may or may not get be assigned a new value. Let FlankUpdate figure it out.
    
        --execute deletes for both beforextstamp and afterxtstamp
        --then update flank
    
        delete *
        from flank
        where beforextstamp between (
                        select min(beforextstamp)
                        from flank
                        where beforextstamp >= '3/16/2009 10:00:46 AM'
                        ) and (
                        select max(beforextstamp)
                        from flank
                        where beforextstamp <= '3/16/2009 10:00:46 AM'
                        );
    
        delete *
        from flank
        where afterxtstamp between (
                        select min(afterxtstamp)
                        from flank
                        where afterxtstamp >= '3/16/2009 10:00:46 AM'
                        ) and (
                        select max(afterxtstamp)
                        from flank
                        where afterxtstamp <= '3/16/2009 10:00:46 AM'
                        );
    
        execute FlankUpdateLoop
    
    --*** Final Report Query***--
        --Should execute without issues including 'deferred execution' problem.
        --Add filters as needed.
        select FirstTable.XmitID
            ,FirstTable.ReceivID
            ,ReceiverDetails.Lat
            ,ReceiverDetails.Lon
            ,BeforeTable.Latitude * (1 - ((FirstTable.RecTStamp - BeforeXTStamp) / (AfterXTStamp - BeforeXTStamp))) + AfterTable.Latitude * ((FirstTable.RecTStamp - BeforeXTStamp) / (AfterXTStamp - BeforeXTStamp)) as Xmit_Lat
            ,BeforeTable.Longitude * (1 - ((FirstTable.RecTStamp - BeforeXTStamp) / (AfterXTStamp - BeforeXTStamp))) + AfterTable.Longitude * ((FirstTable.RecTStamp - BeforeXTStamp) / (AfterXTStamp - BeforeXTStamp)) as Xmit_Lon
            ,FirstTable.RecTStamp as RecTStamp_basic
        from (((
            FirstTable
        inner join Flank on FirstTable.RecTStamp = Flank.RecTStamp)
        inner join SecondTable as BeforeTable on Flank.BeforeXTStamp = BeforeTable.XTStamp)
        inner join SecondTable as AfterTable on Flank.AfterXTStamp = AfterTable.XTStamp)
        inner join ReceiverDetails on FirstTable.ReceivID = ReceiverDetails.ReceivID
        order by FirstTable.RecTStamp;
    
    • 2

relate perguntas

  • Posso automatizar a instrução "on" das consultas MySQL?

  • Usando uma consulta de parâmetro para gerar gráficos em relatórios do MS Access

  • Consultar o banco de dados Linked Access no compartilhamento de rede via SQL Job Agent

  • Qual é a diferença entre um INNER JOIN e um OUTER JOIN?

  • Como é a saída de uma instrução JOIN?

Sidebar

Stats

  • Perguntas 205573
  • respostas 270741
  • best respostas 135370
  • utilizador 68524
  • Highest score
  • respostas
  • Marko Smith

    conectar ao servidor PostgreSQL: FATAL: nenhuma entrada pg_hba.conf para o host

    • 12 respostas
  • Marko Smith

    Como fazer a saída do sqlplus aparecer em uma linha?

    • 3 respostas
  • Marko Smith

    Selecione qual tem data máxima ou data mais recente

    • 3 respostas
  • Marko Smith

    Como faço para listar todos os esquemas no PostgreSQL?

    • 4 respostas
  • Marko Smith

    Listar todas as colunas de uma tabela especificada

    • 5 respostas
  • Marko Smith

    Como usar o sqlplus para se conectar a um banco de dados Oracle localizado em outro host sem modificar meu próprio tnsnames.ora

    • 4 respostas
  • Marko Smith

    Como você mysqldump tabela (s) específica (s)?

    • 4 respostas
  • Marko Smith

    Listar os privilégios do banco de dados usando o psql

    • 10 respostas
  • Marko Smith

    Como inserir valores em uma tabela de uma consulta de seleção no PostgreSQL?

    • 4 respostas
  • Marko Smith

    Como faço para listar todos os bancos de dados e tabelas usando o psql?

    • 7 respostas
  • Martin Hope
    Jin conectar ao servidor PostgreSQL: FATAL: nenhuma entrada pg_hba.conf para o host 2014-12-02 02:54:58 +0800 CST
  • Martin Hope
    Stéphane Como faço para listar todos os esquemas no PostgreSQL? 2013-04-16 11:19:16 +0800 CST
  • Martin Hope
    Mike Walsh Por que o log de transações continua crescendo ou fica sem espaço? 2012-12-05 18:11:22 +0800 CST
  • Martin Hope
    Stephane Rolland Listar todas as colunas de uma tabela especificada 2012-08-14 04:44:44 +0800 CST
  • Martin Hope
    haxney O MySQL pode realizar consultas razoavelmente em bilhões de linhas? 2012-07-03 11:36:13 +0800 CST
  • Martin Hope
    qazwsx Como posso monitorar o andamento de uma importação de um arquivo .sql grande? 2012-05-03 08:54:41 +0800 CST
  • Martin Hope
    markdorison Como você mysqldump tabela (s) específica (s)? 2011-12-17 12:39:37 +0800 CST
  • Martin Hope
    Jonas Como posso cronometrar consultas SQL usando psql? 2011-06-04 02:22:54 +0800 CST
  • Martin Hope
    Jonas Como inserir valores em uma tabela de uma consulta de seleção no PostgreSQL? 2011-05-28 00:33:05 +0800 CST
  • Martin Hope
    Jonas Como faço para listar todos os bancos de dados e tabelas usando o psql? 2011-02-18 00:45:49 +0800 CST

Hot tag

sql-server mysql postgresql sql-server-2014 sql-server-2016 oracle sql-server-2008 database-design query-performance sql-server-2017

Explore

  • Início
  • Perguntas
    • Recentes
    • Highest score
  • tag
  • help

Footer

AskOverflow.Dev

About Us

  • About Us
  • Contact Us

Legal Stuff

  • Privacy Policy

Language

  • Pt
  • Server
  • Unix

© 2023 AskOverflow.DEV All Rights Reserve