如何捕获Postgresql中的更改?

时间:2019-08-02 11:37:59

标签: postgresql apache-kafka debezium oracle-cdc

我遵循了以下步骤:

# Run postgres instance
docker run --name postgres -p 5000:5432 debezium/postgres

# Run zookeeper instance
docker run -it --name zookeeper -p 2181:2181 -p 2888:2888 -p 3888:3888 debezium/zookeeper

# Run kafka instance
docker run -it --name kafka -p 9092:9092 --link zookeeper:zookeeper debezium/kafka

# Run kafka connect
docker run -it --name connect -p 8083:8083 -e GROUP_ID=1 -e CONFIG_STORAGE_TOPIC=my-connect-configs -e OFFSET_STORAGE_TOPIC=my-connect-offsets -e ADVERTISED_HOST_NAME=$(echo $DOCKER_HOST | cut -f3  -d'/' | cut -f1 -d':') --link zookeeper:zookeeper --link postgres:postgres --link kafka:kafka debezium/connect

# Open psql console
psql -h localhost -p 5000 -U postgres
CREATE DATABASE inventory;
CREATE TABLE dumb_table(id SERIAL PRIMARY KEY, name VARCHAR);


# Create connector using kafka connect
curl -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d '
{
    "name": "inventory-connector",
    "config": {
        "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
        "tasks.max": "1",
        "database.hostname": "postgres",
        "database.port": "5432",
        "database.user": "postgres",
        "database.password": "postgres",
        "database.dbname" : "inventory",
        "database.server.name": "dbserver1",
        "database.whitelist": "inventory",
        "database.history.kafka.bootstrap.servers": "kafka:9092",
        "database.history.kafka.topic": "schema-changes.inventory"
    }
}' | jq

# Verify created
curl -H "Accept:application/json" localhost:8083/connectors/ | jq

# Verify configuration
curl -X GET -H "Accept:application/json" localhost:8083/connectors/inventory-connector | jq


# Start a console viewer on kafka

docker run -it --name watcher --rm --link zookeeper:zookeeper debezium/kafka watch-topic -a -k dbserver1.public.dumb_table
# Verify the existence of replication slot in postgres 
SELECT slot_name, plugin, slot_type, database, active, restart_lsn, confirmed_flush_lsn 
FROM pg_replication_slots;

但是在这些步骤结束时,我只在debezium终端中看到错误日志消息。如何查看所有日志消息?

1 个答案:

答案 0 :(得分:0)

打开psql控制台

psql -h localhost -p 5000 -U postgres

创建数据库库存;

此语句之后,您需要连接到刚才创建的清单数据库,因为postgres是默认数据库。

在psql提示符下输入:

\c inventory

这将连接到库存数据库。

现在,建立连接后,请如下所示创建表:

CREATE TABLE dumb_table(id SERIAL PRIMARY KEY, name VARCHAR);

现在在同一数据库(即广告资源)下的同一表上发出一些插入命令

INSERT INTO dumb_table VALUES (1, 'VALUE1');

您将能够在使用者控制台中看到更改。

在创建debezium连接器以侦听库存数据库中的更改时。因此,它仅侦听在库存数据库下创建的表的更改(插入/更新/删除)。