主页 / user-4666

adinas's questions

Martin Hope
adinas
Asked: 2024-12-17 17:58:10 +0800 CST
  • 6

我认为 Cassandra 4.1.7 表中的墓碑不会被删除。

该表的 gc_grace_seconds = 345600

即使在运行之后nodetool garbagecollect（删除后远晚于 4 天），如果我运行，nodetool tablestats我会得到以下结果：

在此处输入图片描述

执行垃圾收集后，磁盘大小也不会减少。

我不确定我过去是否跑过nodetool compact（我想这意味着我需要继续使用它？）

附带问题，如何让 tablestats 返回更新的信息（而不是从 2024-12-12 开始）

谢谢

cassandra
Martin Hope
adinas
Asked: 2024-11-03 18:26:05 +0800 CST
  • 6

我们有一个 2 节点集群（我知道 3 是建议的最小值），它已经运行 Cassandra 3 和 4 多年，没有发生任何事故。

上周我们将两个节点都升级到了 Cassandra 5。

自那时起，节点之间似乎出现了流言蜚语和沟通问题。尤其是在更新架构时（例如创建或删除表时）。

这是在进行架构更改（删除表）之后立即出现的调试日志：

INFO  [Native-Transport-Requests-1] 2024-11-03 09:48:04,021 Keyspace.java:379 - Creating replication strategy r_internal params KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy, replication_factor=2}}
DEBUG [Native-Transport-Requests-1] 2024-11-03 09:48:04,064 MigrationCoordinator.java:430 - Reported schema a47919aa-79b3-362c-b023-d8ac89671885 at endpoint /10.10.2.27:7000
DEBUG [MigrationStage:1] 2024-11-03 09:48:04,070 MigrationCoordinator.java:749 - Pushing schema mutations: [Mutation(keyspace='system_schema', key='72617976656e5f696e7465726e616c', modifications=[
  [system_schema.columns] key=r_internal partition_deletion=deletedAt=-9223372036854775808, localDeletion=9223372036854775807 columns=[[] | []]
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, column_name=cdate |
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, column_name=device_id |
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, column_name=event_timestamp |
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, column_name=info |
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, column_name=input |
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, column_name=node_id |
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, column_name=output |
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, column_name=server |
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, column_name=unique_flow_process |
  [system_schema.indexes] key=r_internal partition_deletion=deletedAt=-9223372036854775808, localDeletion=9223372036854775807 columns=[[] | []]
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, index_name=p1489_log_node_input_output_unique_flow_process_idx |
  [system_schema.keyspaces] key=r_internal partition_deletion=deletedAt=-9223372036854775808, localDeletion=9223372036854775807 columns=[[] | [durable_writes replication]]
    Row[info=[ts=1730627238890000] ]: EMPTY | [durable_writes=true ts=1730627238890000], [replication=0000000200000005636c6173730000002b6f72672e6170616368652e63617373616e6472612e6c6f6361746f722e53696d706c655374726174656779000000127265706c69636174696f6e5f666163746f720000000132 ts=1730627238890000]
  [system_schema.tables] key=r_internal partition_deletion=deletedAt=-9223372036854775808, localDeletion=9223372036854775807 columns=[[] | []]
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output |
])]
DEBUG [MigrationStage:1] 2024-11-03 09:48:04,071 MigrationCoordinator.java:757 - Pushing schema mutations to /10.10.2.28:7000: [Mutation(keyspace='system_schema', key='72617976656e5f696e7465726e616c', modifications=[
  [system_schema.columns] key=r_internal partition_deletion=deletedAt=-9223372036854775808, localDeletion=9223372036854775807 columns=[[] | []]
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, column_name=cdate |
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, column_name=device_id |
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, column_name=event_timestamp |
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, column_name=info |
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, column_name=input |
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, column_name=node_id |
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, column_name=output |
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, column_name=server |
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, column_name=unique_flow_process |
  [system_schema.indexes] key=r_internal partition_deletion=deletedAt=-9223372036854775808, localDeletion=9223372036854775807 columns=[[] | []]
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output, index_name=p1489_log_node_input_output_unique_flow_process_idx |
  [system_schema.keyspaces] key=r_internal partition_deletion=deletedAt=-9223372036854775808, localDeletion=9223372036854775807 columns=[[] | [durable_writes replication]]
    Row[info=[ts=1730627238890000] ]: EMPTY | [durable_writes=true ts=1730627238890000], [replication=0000000200000005636c6173730000002b6f72672e6170616368652e63617373616e6472612e6c6f6361746f722e53696d706c655374726174656779000000127265706c69636174696f6e5f666163746f720000000132 ts=1730627238890000]
  [system_schema.tables] key=r_internal partition_deletion=deletedAt=-9223372036854775808, localDeletion=9223372036854775807 columns=[[] | []]
    Row[info=[ts=-9223372036854775808] del=deletedAt=1730627238890000, localDeletion=1730627238 ]: table_name=p1489_log_node_input_output |
])]
ERROR [GossipStage:1] 2024-11-03 09:48:04,187 JVMStabilityInspector.java:70 - Exception in thread Thread[GossipStage:1,5,GossipStage]
java.lang.RuntimeException: java.lang.AssertionError
        at org.apache.cassandra.net.InboundSink.accept(InboundSink.java:108)
        at org.apache.cassandra.net.InboundSink.accept(InboundSink.java:45)
        at org.apache.cassandra.net.InboundMessageHandler$ProcessMessage.run(InboundMessageHandler.java:430)
        at org.apache.cassandra.concurrent.ExecutionFailure$1.run(ExecutionFailure.java:133)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.AssertionError: null
        at org.apache.cassandra.db.TypeSizes.sizeof(TypeSizes.java:44)
        at org.apache.cassandra.gms.VersionedValue$VersionedValueSerializer.serializedSize(VersionedValue.java:365)
        at org.apache.cassandra.gms.VersionedValue$VersionedValueSerializer.serializedSize(VersionedValue.java:343)
        at org.apache.cassandra.gms.EndpointStateSerializer.serializedSize(EndpointState.java:344)
        at org.apache.cassandra.gms.EndpointStateSerializer.serializedSize(EndpointState.java:300)
        at org.apache.cassandra.gms.GossipDigestAckSerializer.serializedSize(GossipDigestAck.java:96)
        at org.apache.cassandra.gms.GossipDigestAckSerializer.serializedSize(GossipDigestAck.java:61)
        at org.apache.cassandra.net.Message$Serializer.payloadSize(Message.java:1088)
        at org.apache.cassandra.net.Message.payloadSize(Message.java:1131)
        at org.apache.cassandra.net.Message$Serializer.serializedSize(Message.java:769)
        at org.apache.cassandra.net.Message.serializedSize(Message.java:1111)
        at org.apache.cassandra.net.OutboundConnections.connectionTypeFor(OutboundConnections.java:215)
        at org.apache.cassandra.net.OutboundConnections.connectionFor(OutboundConnections.java:207)
        at org.apache.cassandra.net.OutboundConnections.enqueue(OutboundConnections.java:96)
        at org.apache.cassandra.net.MessagingService.doSend(MessagingService.java:473)
        at org.apache.cassandra.net.OutboundSink.accept(OutboundSink.java:70)
        at org.apache.cassandra.net.MessagingService.send(MessagingService.java:462)
        at org.apache.cassandra.net.MessagingService.send(MessagingService.java:437)
        at org.apache.cassandra.gms.GossipDigestSynVerbHandler.doVerb(GossipDigestSynVerbHandler.java:110)
        at org.apache.cassandra.net.InboundSink.lambda$new$0(InboundSink.java:78)
        at org.apache.cassandra.net.InboundSink.accept(InboundSink.java:97)
        ... 7 common frames omitted
DEBUG [GossipStage:1] 2024-11-03 09:48:04,187 HeapUtils.java:133 - Heap dump creation on uncaught exceptions is disabled.
DEBUG [GossipStage:1] 2024-11-03 09:48:04,187 HeapUtils.java:133 - Heap dump creation on uncaught exceptions is disabled.

线程 Thread[GossipStage:1,5,GossipStage] 中的异常Exception不断重复出现。

此时，运行nodetool status显示，每个节点都认为另一个节点已关闭 在此处输入图片描述

我通常需要耗尽并重新启动两个节点（我一个接一个地执行此操作），以便它们互相看到并显示它们的模式是同步的。

以下是nodetool gossipinfo的输出（重新启动节点后）

/10.10.2.27
  generation:1730627829
  heartbeat:939
  STATUS:23:NORMAL,-1656170799351181049
  LOAD:898:2.029075780322E12
  SCHEMA:17:a47919aa-79b3-362c-b023-d8ac89671885
  DC:13:dc1
  RACK:15:rack1
  RELEASE_VERSION:6:5.0.2
  INTERNAL_IP:11:10.10.2.27
  RPC_ADDRESS:5:10.10.2.27
  NET_VERSION:2:12
  HOST_ID:3:8be5925c-0436-4bc7-bd4c-0fb5dd8fc9e5
  RPC_READY:35:true
  INTERNAL_ADDRESS_AND_PORT:9:10.10.2.27:7000
  NATIVE_ADDRESS_AND_PORT:4:10.10.2.27:9042
  STATUS_WITH_PORT:22:NORMAL,-1656170799351181049
  SSTABLE_VERSIONS:7:big-nb
  TOKENS:21:<hidden>
/10.10.2.28
  generation:1730628138
  heartbeat:619
  LOAD:586:2.215579324507E12
  SCHEMA:17:a47919aa-79b3-362c-b023-d8ac89671885
  DC:13:dc1
  RACK:15:rack1
  RELEASE_VERSION:6:5.0.2
  NET_VERSION:2:12
  HOST_ID:3:c9e8a4f8-c7b2-4cac-994a-e8de50c9f358
  RPC_READY:35:true
  INTERNAL_ADDRESS_AND_PORT:9:10.10.2.28:7000
  NATIVE_ADDRESS_AND_PORT:4:10.10.2.28:9042
  STATUS_WITH_PORT:22:NORMAL,-2024247660995288657
  SSTABLE_VERSIONS:7:big-nb
  TOKENS:21:<hidden>

这里从第二个节点开始

/10.10.2.27
  generation:1730627829
  heartbeat:906
  LOAD:898:2.029075780322E12
  SCHEMA:17:a47919aa-79b3-362c-b023-d8ac89671885
  DC:13:dc1
  RACK:15:rack1
  RELEASE_VERSION:6:5.0.2
  NET_VERSION:2:12
  HOST_ID:3:8be5925c-0436-4bc7-bd4c-0fb5dd8fc9e5
  RPC_READY:35:true
  INTERNAL_ADDRESS_AND_PORT:9:10.10.2.27:7000
  NATIVE_ADDRESS_AND_PORT:4:10.10.2.27:9042
  STATUS_WITH_PORT:22:NORMAL,-1656170799351181049
  SSTABLE_VERSIONS:7:big-nb
  TOKENS:21:<hidden>
/10.10.2.28
  generation:1730628138
  heartbeat:584
  STATUS:23:NORMAL,-2024247660995288657
  LOAD:524:2.215537773956E12
  SCHEMA:17:a47919aa-79b3-362c-b023-d8ac89671885
  DC:13:dc1
  RACK:15:rack1
  RELEASE_VERSION:6:5.0.2
  INTERNAL_IP:11:10.10.2.28
  RPC_ADDRESS:5:10.10.2.28
  NET_VERSION:2:12
  HOST_ID:3:c9e8a4f8-c7b2-4cac-994a-e8de50c9f358
  RPC_READY:35:true
  INTERNAL_ADDRESS_AND_PORT:9:10.10.2.28:7000
  NATIVE_ADDRESS_AND_PORT:4:10.10.2.28:9042
  STATUS_WITH_PORT:22:NORMAL,-2024247660995288657
  SSTABLE_VERSIONS:7:big-nb
  TOKENS:21:<hidden>

当他们遇到问题时，这是在 10.10.2.28 上运行时nodetool gossinfo显示的内容：

/10.10.2.27
  generation:1730626480
  heartbeat:810
  LOAD:771:2.029304184276E12
  SCHEMA:17:9bcfb867-92ca-3144-92d7-da5fed3359d0
  DC:13:dc1
  RACK:15:rack1
  RELEASE_VERSION:6:5.0.2
  NET_VERSION:2:12
  HOST_ID:3:8be5925c-0436-4bc7-bd4c-0fb5dd8fc9e5
  RPC_READY:35:true
  INTERNAL_ADDRESS_AND_PORT:9:10.10.2.27:7000
  NATIVE_ADDRESS_AND_PORT:4:10.10.2.27:9042
  STATUS_WITH_PORT:22:NORMAL,-1656170799351181049
  SSTABLE_VERSIONS:7:big-nb
  TOKENS:21:<hidden>
/10.10.2.28
  generation:1730625184
  heartbeat:2523
  STATUS:23:NORMAL,-2024247660995288657
  LOAD:2513:2.214202366182E12
  SCHEMA:2243:a47919aa-79b3-362c-b023-d8ac89671885
  DC:13:dc1
  RACK:15:rack1
  RELEASE_VERSION:6:5.0.2
  INTERNAL_IP:11:10.10.2.28
  RPC_ADDRESS:5:10.10.2.28
  NET_VERSION:2:12
  HOST_ID:3:c9e8a4f8-c7b2-4cac-994a-e8de50c9f358
  RPC_READY:35:true
  INTERNAL_ADDRESS_AND_PORT:9:10.10.2.28:7000
  NATIVE_ADDRESS_AND_PORT:4:10.10.2.28:9042
  STATUS_WITH_PORT:22:NORMAL,-2024247660995288657
  SSTABLE_VERSIONS:7:big-nb
  INDEX_STATUS:2198:{"r_internal.p1643_log_node_input_output_unique_flow_process_idx":"BUILD_SUCCEEDED","r_internal.p1720_log_node_input_output_unique_flow_process_idx":"BUILD_SUCCEEDED","r_internal.p1479_log_node_input_output_unique_flow_process_idx":"BUILD_SUCCEEDED"...}
  TOKENS:21:<hidden>
cassandra
Martin Hope
adinas
Asked: 2024-10-28 22:43:50 +0800 CST
  • 6

我已经在我的 Linux 机器（Ubutnu）上安装了 Cassandra。

我安装了 OpenJDK 11，然后安装了 JDK 17 并将其设置为默认值并重新启动了 Cassandra。

Java 精选

我可以看到 /var/log/cassandra/system.log 中正在使用新版本 Cassandra 日志

一切正常，但我不清楚使用 17 是否被视为“实验性的”或是否比 11 更可取？

cassandra
Martin Hope
adinas
Asked: 2024-01-25 17:50:11 +0800 CST
  • 5

摘要：我有一个 2 节点集群，如果其中一个节点发生故障，我将无法登录。我收到错误。

Connection error: ('Unable to connect to any servers', {'192.168.1.104:9042': AuthenticationFailed('Failed to authenticate to 192.168.1.104:9042: Error from server: code=0100 [Bad credentials] message="Unable to perform authentication: Cannot achieve consistency level LOCAL_QUORUM"')})
  • 我正在使用我创建的角色（不是 SUEPRUSER，因此 SUPERUSER 始终使用 QUORUM 的问题不应适用于此。根据我的阅读，它应该使用 LOCAL_ONE 但不是！
  • 我特意将system_auth键空间复制设置为 2，因为建议这样做不会出现单点故障。

这是一个屏幕截图，显示我没有使用 cassandra SUPERUSER 但收到错误： 在此输入图像描述

我希望即使一个节点发生故障，登录也能继续工作。

cassandra