福永网站建设公司哪家好,wordpress 多菜单,2018什么语言做网站比较好,海宁公司做网站目录 一、docker部署RocketMQ1、简易说明2、docker拉取RocketMQ镜像\RocketMQ控制台3、获取RocketMQ配置文件4、RocketMQ配置文件描述5、docker启动RocketMQ6、进入RocketMQ控制台 二、填坑经验错误一: connect to 172.17.0.3:10909 failed错误二: maybe your broker m… 目录 一、docker部署RocketMQ1、简易说明2、docker拉取RocketMQ镜像\RocketMQ控制台3、获取RocketMQ配置文件4、RocketMQ配置文件描述5、docker启动RocketMQ6、进入RocketMQ控制台 二、填坑经验错误一: connect to 172.17.0.3:10909 failed错误二: maybe your broker machine memory too small 本人安装版本最新版(rocketmq-4.4.0),以下均对应4.4.0版本一、docker部署RocketMQ
1、简易说明
在RocketMQ中有三个关键组件NamesrvName Server、Broker和Console-ng管理控制台。 NamesrvName ServerNamesrv是RocketMQ的命名服务负责管理整个RocketMQ集群的路由信息。每个RocketMQ集群中都至少需要一个Namesrv实例。它维护了Broker的网络信息、Topic的路由规则以及Consumer的消费进度等元数据并提供给Producer和Consumer使用。 BrokerBroker是RocketMQ的消息存储和处理节点负责存储消息、处理消息的读写请求和转发消息等功能。在RocketMQ集群中可以有多个Broker实例各个Broker通过与Namesrv交互来维护消息的元数据和路由信息以实现高可用、负载均衡的消息传输。 Console-ng管理控制台Console-ng是RocketMQ官方提供的管理控制台用于管理和监控RocketMQ集群。它提供了图形化界面可以进行Topic、Consumer等的配置管理、消息查询与追踪、监控指标展示等操作。Console-ng对于集群的监控和运维非常有用。
这三个组件共同构成了RocketMQ的核心架构并协同工作以实现高可用、高性能的消息传输和数据管理。您可以通过启动Namesrv、Broker来搭建一个RocketMQ集群并使用Console-ng进行集群的管理与监控。
2、docker拉取RocketMQ镜像\RocketMQ控制台
拉取最新的RocketMQ如果下载指定版本可以去docker官网查看 注Namesrv、Broker均采用rocketmqinc/rocketmq同一个镜像
# 最新(rocketmq)
docker pull rocketmqinc/rocketmq
# 指定版本号(rocketmq)
docker pull rocketmqinc/rocketmq:版本号# 最新(RocketMQ控制台)
docker pull pangliang/rocketmq-console-ng3、获取RocketMQ配置文件
可以启动RocketMQ然后从docker容器中拷贝出配置文件拷出配置文件后启动的容器就可以删除了
docker run -d --name rmqnamesrv -p 9876:9876 rocketmqinc/rocketmq:latest sh mqnamesrv
# 进入容器(用于进入容器找到broker.conf 的位置)
docker exec -it 容器ID /bin/bash
# 从容器中下载文件到虚拟机
docker cp 容器ID:/opt/rocketmq-4.4.0/conf/broker.conf 虚拟机路径4、RocketMQ配置文件描述
ongPollingEnabletrue
offsetCheckInSlavefalse
# nameServer地址分号分割
namesrvAddr172.16.234.150:9876
fetchNamesrvAddrByAddressServerfalse
#是否允许 Broker 自动创建订阅组建议线下开启线上关闭
autoCreateSubscriptionGrouptrue
#是否允许 Broker 自动创建Topic建议线下开启线上关闭
autoCreateTopicEnabletrue
sendThreadPoolQueueCapacity100000
clusterTopicEnabletrue
filterServerNums1
pullMessageThreadPoolNums20
# broker名字名字可重复,为了管理,每个master起一个名字,他的slave同他,eg:Amaster叫broker-a,他的slave也叫broker-a
brokerNameknBroker
#rocketmqHome/usr/local/alibaba-rocketmq/
sendMessageThreadPoolNums24
# 0 表示 Master0 表示 Slave
brokerId0
brokerIP1172.16.234.150
brokerTopicEnabletrue
brokerPermission6
shortPollingTimeMills1000
clientManageThreadPoolNums16
adminBrokerThreadPoolNums16
flushConsumerOffsetInterval5000
flushConsumerOffsetHistoryInterval60000
# 在发送消息时自动创建服务器不存在的topic默认创建的队列数
defaultTopicQueueNums8
rejectTransactionMessagefalse
notifyConsumerIdsChangedEnabletrue
pullThreadPoolQueueCapacity100000
# # 所属集群名字
brokerClusterNameDefaultCluster
putMsgIndexHightWater600000
maxTransferBytesOnMessageInDisk65536
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio75
checkCRCOnRecovertrue
haSlaveFallbehindMax268435
deleteConsumeQueueFilesInterval100
cleanResourceInterval10000
maxMsgsNumBatch64
flushConsumeQueueLeastPages2
syncFlushTimeout5000
#删除文件时间点默认凌晨 4点
deleteWhen04
#Broker 的角色
brokerRoleASYNC_MASTER
destroyMapedFileIntervalForcibly120000
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog1073741824
haSendHeartbeatInterval5000
#刷盘方式
flushDiskTypeASYNC_FLUSH
cleanFileForciblyEnabletrue
haHousekeepingInterval20000
redeleteHangedFileInterval120000
#限制的消息大小
maxMessageSize524288
flushCommitLogTimedfalse
haMasterAddress
maxTransferCountOnMessageInDisk4
flushIntervalCommitLog1000
#文件保留时间默认 48 小时
fileReservedTime72
flushCommitLogThoroughInterval10000
maxHashSlotNum5000
maxIndexNum20000
messageIndexEnabletrue
#存储路径
storePathRootDir/root/store
#commitLog 存储路径
storePathCommitLog/root/store/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue/root/store/consumequeue
#消息索引存储路径
storePathIndex/root/store/index
haListenPort10912
flushDelayOffsetInterval10000
haTransferBatchSize32768
deleteCommitLogFilesInterval100
maxTransferBytesOnMessageInMemory262144
accessMessageInMemoryMaxRatio40
flushConsumeQueueThoroughInterval60000
flushIntervalConsumeQueue1000
maxTransferCountOnMessageInMemory32
messageIndexSafefalse
#ConsumeQueue每个文件默认存30W条根据业务情况调整
mapedFileSizeConsumeQueue6000000
messageDelayLevel1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h
flushCommitLogLeastPages4
serverChannelMaxIdleTimeSeconds120
#Broker 对外服务的监听端口
listenPort10911
serverCallbackExecutorThreads0
serverAsyncSemaphoreValue64
serverSocketSndBufSize131072
serverSelectorThreads3
serverPooledByteBufAllocatorEnablefalse
serverWorkerThreads8
serverSocketRcvBufSize131072
serverOnewaySemaphoreValue256
clientWorkerThreads4
connectTimeoutMillis3000
clientSocketRcvBufSize131072
clientOnewaySemaphoreValue2048
clientChannelMaxIdleTimeSeconds120
clientPooledByteBufAllocatorEnablefalse
clientAsyncSemaphoreValue2048
channelNotActiveInterval60000
clientCallbackExecutorThreads2
clientSocketSndBufSize1310725、docker启动RocketMQ
以下将配置文件、日志、存储均挂载在本地
# 1、namesrv
docker run -d -p 9876:9876 \
-v /mydata/rocketmq/namesrv/logs:/root/logs \
-v /mydata/rocketmq/namesrv/store:/root/store \
-v /mydata/rocketmq/conf/broker.conf:/opt/rocketmq-4.4.0/conf/broker.conf \
--name rmqnamesrv \
rocketmqinc/rocketmq:latest sh mqnamesrv# 2、broker
docker run -d -p 10911:10911 -p 10909:10909 \
-v /mydata/rocketmq/broker/logs:/root/logs \
-v /mydata/rocketmq/broker/store:/root/store \
-v /mydata/rocketmq/conf/broker.conf:/opt/rocketmq-4.4.0/conf/broker.conf \
--name rmqbroker \
--add-host namesrv:172.16.234.150 \
-e NAMESRV_ADDRnamesrv:9876 \
rocketmqinc/rocketmq:latest \
sh mqbroker -n namesrv:9876 \
-c /opt/rocketmq-4.4.0/conf/broker.conf autoCreateTopicEnabletrue# 3、Console-ng
docker run --name rocketmq-console \
-e JAVA_OPTS-Drocketmq.namesrv.addr172.16.234.150:9876 \
-Dcom.rocketmq.sendMessageWithVIPChannelfalse \
-p 8080:8080 -t styletang/rocketmq-console-ng6、进入RocketMQ控制台
地址(IProcketmq-console端口)http://172.16.234.150:8080/#/
二、填坑经验
错误一: connect to 172.17.0.3:10909 failed
1、在启动Java项目后发送MQ消息是报以下错误 2、RocketMQ控制台集群地址显示为docker分配的IP
进入RocketMQ控制台控制台显示的集群地址为172.17.0.3:10909并非172.16.234.150:10909172.17.0.3实际为docker内部分配的ID需要此IP修改为虚拟机的IP
为了解决以上问题使用docker创建了一个网格【docker network create rocketmq-net docker run -d --network rocketmq-net …】但是没有解决此问题实际导致此问题的是RocketMQ配置文件请核查RocketMQ配置文件或者采用上方提供的配置文件
com.himyidea.framework.mq.MQRuntimeException: EC 900101: MSG 900101 | msgMQ Client Failureat com.himyidea.framework.mq.producer.impl.GeneralMQProducer.doSend(GeneralMQProducer.java:130)at com.himyidea.framework.mq.producer.impl.GeneralMQProducer.sendMessage(GeneralMQProducer.java:105)at java.lang.Thread.run(Thread.java:748)
Caused by: com.alibaba.rocketmq.client.exception.MQClientException: Send [3] times, still failed, cost [9073]ms, Topic: report_data_topic, BrokersSent: [broker-a, broker-a, broker-a]
See http://docs.aliyun.com/cn#/pub/ons/faq/exceptionssend_msg_failed for further details.at com.alibaba.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendDefaultImpl(DefaultMQProducerImpl.java:522)at com.alibaba.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1030)at com.alibaba.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:989)at com.alibaba.rocketmq.client.producer.DefaultMQProducer.send(DefaultMQProducer.java:90)at com.himyidea.framework.mq.producer.impl.GeneralMQProducer.doSend(GeneralMQProducer.java:126)... 85 more
Caused by: com.alibaba.rocketmq.remoting.exception.RemotingConnectException: connect to 172.17.0.3:10909 failedat com.alibaba.rocketmq.remoting.netty.NettyRemotingClient.invokeSync(NettyRemotingClient.java:360)at com.alibaba.rocketmq.client.impl.MQClientAPIImpl.sendMessageSync(MQClientAPIImpl.java:267)at com.alibaba.rocketmq.client.impl.MQClientAPIImpl.sendMessage(MQClientAPIImpl.java:251)at com.alibaba.rocketmq.client.impl.MQClientAPIImpl.sendMessage(MQClientAPIImpl.java:214)at com.alibaba.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendKernelImpl(DefaultMQProducerImpl.java:671)at com.alibaba.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendDefaultImpl(DefaultMQProducerImpl.java:440)错误二: maybe your broker machine memory too small
内存不足以为是docker启动命令中未传内存信息实际是虚拟机可用内存空间不足
Caused by: com.alibaba.rocketmq.client.exception.MQBrokerException: CODE: 14 DESC: service not available now, maybe disk full, CL: 0.99 CQ: -1.00 INDEX: -1.00, maybe your broker machine memory too small.
