apache-kafka - KSQL 流主题存储在哪里?

运行命令:

`λ oc exec -it ms-ksql-db-server-deployment-76f454b75-mjfbh bash`

[appuser@ms-ksql-db-server-deployment-76f454b75-mjfbh ~]$ `ksql`
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.

              ===========================================
              =       _              _ ____  ____       =
              =      | | _____  __ _| |  _ \| __ )      =
              =      | |/ / __|/ _` | | | | |  _ \      =
              =      |   <\__ \ (_| | | |_| | |_) |     =
              =      |_|\_\___/\__, |_|____/|____/      =
              =                   |_|                   =
              =        The Database purpose-built       =
              =        for stream processing apps       =
              ===========================================

Copyright 2017-2022 Confluent Inc.

CLI v0.26.0, Server v0.26.0 located at http://localhost:8088
Server Status: RUNNING

Having trouble? Type 'help' (case-insensitive) for a rundown of how things work!

ksql> CREATE STREAM CLIENT_STREAM WITH (KAFKA_TOPIC='private-import-clients',VALUE_FORMAT='AVRO');

 Message
----------------
 Stream created
----------------
ksql> SET 'auto.offset.reset'='earliest';
Successfully changed local property 'auto.offset.reset' to 'earliest'. Use the UNSET command to revert your change.
ksql> SHOW STREAMS;

 Stream Name         | Kafka Topic                 | Key Format | Value Format | Windowed
------------------------------------------------------------------------------------------
 CLIENT_STREAM       | private-import-clients      | KAFKA      | AVRO         | false
 KSQL_PROCESSING_LOG | default_ksql_processing_log | KAFKA      | JSON         | false
------------------------------------------------------------------------------------------
ksql> SELECT * FROM CLIENT_STREAM EMIT CHANGES;

我正在从一个主题 private-import-clients 编写 KSQL Streams,我假设我创建的 Stream 的结果将给 origin 一个主题 CLIENT_STREAM 或其他名称。这是对的吗?如果是这样,KSQL stores 那个主题 CLIENT_STREAM 在哪里?我有一个 Spring 启动应用程序,它将使用主题 CLIENT_STREAM 。

回答1

CREATE STREAM 语句是 DDL 语句,因此操作是更新 ksqlDB 元数据。

每个 ksqlDB 服务器都有一个内部的内存中元数据 store 或元存储,它在接收 DDL 语句时构建。 Metastore 是内存中的 map。对于每个新的 DDL 语句,ksqlDB 引擎都会向元存储添加一个条目。

ksqlDB 引擎将 DML 语句转换为 Kafka Streams 应用程序

https://docs.ksqldb.io/en/latest/operate-and-deploy/how-it-works/解释了 KSQL 如何创建逻辑、物理计划以及它如何处理查询。

因此它 stores 仅用于 DDL 语句的内存中 map 中的本地元数据,其行为类似于 DML 语句的流。