[alibaba/canal]canal-deployer-1.1.6 监听 RDS binlog,配置发送至阿里云 RocketMq,出现主题分区 数组越界bug,请指教

2023-12-12 854 views
9
Question

canal-deployer-1.1.6 监听 RDS binlog,配置发送至阿里云 RocketMq,出现主题分区 数组越界bug,请指教; 2023-11-30 19:18:15.913 [pool-4-thread-1] ERROR c.a.o.c.c.rocketmq.producer.CanalRocketMQProducer - java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: 0 java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: 0 at com.alibaba.otter.canal.common.utils.ExecutorTemplate.waitForResult(ExecutorTemplate.java:74) ~[canal.common-1.1.6.jar:na] at com.alibaba.otter.canal.connector.rocketmq.producer.CanalRocketMQProducer.send(CanalRocketMQProducer.java:168) ~[na:na] at com.alibaba.otter.canal.server.CanalMQStarter.worker(CanalMQStarter.java:181) [canal.server-1.1.6.jar:na] at com.alibaba.otter.canal.server.CanalMQStarter.access$100(CanalMQStarter.java:24) [canal.server-1.1.6.jar:na] at com.alibaba.otter.canal.server.CanalMQStarter$CanalMQRunnable.run(CanalMQStarter.java:223) [canal.server-1.1.6.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_372] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_372] at java.lang.Thread.run(Thread.java:750) [na:1.8.0_372] Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: 0 at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[na:1.8.0_372] at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[na:1.8.0_372] at com.alibaba.otter.canal.common.utils.ExecutorTemplate.waitForResult(ExecutorTemplate.java:72) ~[canal.common-1.1.6.jar:na] ... 7 common frames omitted Caused by: java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: 0 at com.alibaba.otter.canal.connector.rocketmq.producer.CanalRocketMQProducer.lambda$send$0(CanalRocketMQProducer.java:163) ~[na:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_372] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_372] ... 3 common frames omitted 配置如下: canal.properties 中 rocketmqnameserver address 配置的是 阿里云mrocketq 提供的endpoint接入点.

image

example.properties 分区相关设置如下:

image

回答

9

看不到更原始的堆账,先考虑升级版本1.1.7吧

2

升级到1.1.7 后 将配置 改成如下:

image canal.mq.partitionsNum=1 注释掉了,想依靠 canal.mq.enableDynamicQueuePartition 自动获取分区数,但是会有如下 报错:

2023-12-05 11:38:04.323 [pool-4-thread-1] ERROR c.a.o.c.c.rocketmq.producer.CanalRocketMQProducer - java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.NullPointerException java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.NullPointerException at com.alibaba.otter.canal.common.utils.ExecutorTemplate.waitForResult(ExecutorTemplate.java:74) at com.alibaba.otter.canal.connector.rocketmq.producer.CanalRocketMQProducer.send(CanalRocketMQProducer.java:168) at com.alibaba.otter.canal.connector.core.spi.ProxyCanalMQProducer.send(ProxyCanalMQProducer.java:51) at com.alibaba.otter.canal.server.CanalMQStarter.worker(CanalMQStarter.java:181) at com.alibaba.otter.canal.server.CanalMQStarter.access$100(CanalMQStarter.java:24) at com.alibaba.otter.canal.server.CanalMQStarter$CanalMQRunnable.run(CanalMQStarter.java:223) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.NullPointerException at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at com.alibaba.otter.canal.common.utils.ExecutorTemplate.waitForResult(ExecutorTemplate.java:72) ... 8 common frames omitted Caused by: java.lang.RuntimeException: java.lang.NullPointerException at com.alibaba.otter.canal.connector.rocketmq.producer.CanalRocketMQProducer.lambda$send$0(CanalRocketMQProducer.java:163) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 3 common frames omitted Caused by: java.lang.NullPointerException: null at com.alibaba.otter.canal.connector.rocketmq.producer.CanalRocketMQProducer.send(CanalRocketMQProducer.java:239) at com.alibaba.otter.canal.connector.rocketmq.producer.CanalRocketMQProducer.lambda$send$0(CanalRocketMQProducer.java:161) ... 5 common frames omitted 2023-12-05 11:38:04.326 [pool-4-thread-1] ERROR c.a.o.c.c.rocketmq.producer.CanalRocketMQProducer - java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.NullPointerException java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.NullPointerException at com.alibaba.otter.canal.common.utils.ExecutorTemplate.waitForResult(ExecutorTemplate.java:74) at com.alibaba.otter.canal.connector.rocketmq.producer.CanalRocketMQProducer.send(CanalRocketMQProducer.java:168) at com.alibaba.otter.canal.connector.core.spi.ProxyCanalMQProducer.send(ProxyCanalMQProducer.java:51) at com.alibaba.otter.canal.server.CanalMQStarter.worker(CanalMQStarter.java:181) at com.alibaba.otter.canal.server.CanalMQStarter.access$100(CanalMQStarter.java:24) at com.alibaba.otter.canal.server.CanalMQStarter$CanalMQRunnable.run(CanalMQStarter.java:223) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.NullPointerException at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at com.alibaba.otter.canal.common.utils.ExecutorTemplate.waitForResult(ExecutorTemplate.java:72) ... 8 common frames omitted

8

当我把 注释放开 canal.mq.partitionsNum=1,

image

我猜估计上次报空指针是因为 无法自动获取分区数,然后我又把 canal.mq.partitionsNum=1注释掉了的原因,这次注释放开后,果然不报空指针了。报以下错误: 2023-12-05 12:04:28.366 [pool-4-thread-1] ERROR c.a.o.c.c.rocketmq.producer.CanalRocketMQProducer - java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.apache.rocketmq.client.exception.MQBrokerException: CODE: 3 DESC: request code unsupported, code 320 BROKER: 172.16.16.12:8080 For more information, please visit the url, http://rocketmq.apache.org/docs/faq/ java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.apache.rocketmq.client.exception.MQBrokerException: CODE: 3 DESC: request code unsupported, code 320 BROKER: 172.16.16.12:8080 For more information, please visit the url, http://rocketmq.apache.org/docs/faq/ at com.alibaba.otter.canal.common.utils.ExecutorTemplate.waitForResult(ExecutorTemplate.java:74) at com.alibaba.otter.canal.connector.rocketmq.producer.CanalRocketMQProducer.send(CanalRocketMQProducer.java:168) at com.alibaba.otter.canal.connector.core.spi.ProxyCanalMQProducer.send(ProxyCanalMQProducer.java:51) at com.alibaba.otter.canal.server.CanalMQStarter.worker(CanalMQStarter.java:181) at com.alibaba.otter.canal.server.CanalMQStarter.access$100(CanalMQStarter.java:24) at com.alibaba.otter.canal.server.CanalMQStarter$CanalMQRunnable.run(CanalMQStarter.java:223) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.apache.rocketmq.client.exception.MQBrokerException: CODE: 3 DESC: request code unsupported, code 320 BROKER: 172.16.16.12:8080 For more information, please visit the url, http://rocketmq.apache.org/docs/faq/ at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at com.alibaba.otter.canal.common.utils.ExecutorTemplate.waitForResult(ExecutorTemplate.java:72) ... 8 common frames omitted Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.apache.rocketmq.client.exception.MQBrokerException: CODE: 3 DESC: request code unsupported, code 320 BROKER: 172.16.16.12:8080 For more information, please visit the url, http://rocketmq.apache.org/docs/faq/ at com.alibaba.otter.canal.connector.rocketmq.producer.CanalRocketMQProducer.lambda$send$0(CanalRocketMQProducer.java:163) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 3 common frames omitted Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.apache.rocketmq.client.exception.MQBrokerException: CODE: 3 DESC: request code unsupported, code 320 BROKER: 172.16.16.12:8080 For more information, please visit the url, http://rocketmq.apache.org/docs/faq/ at com.alibaba.otter.canal.common.utils.ExecutorTemplate.waitForResult(ExecutorTemplate.java:74) at com.alibaba.otter.canal.connector.rocketmq.producer.CanalRocketMQProducer.send(CanalRocketMQProducer.java:275) at com.alibaba.otter.canal.connector.rocketmq.producer.CanalRocketMQProducer.lambda$send$0(CanalRocketMQProducer.java:161)