一、簡介
activeMQ是apache開源的一個消息中間件.之前的RPC基本上都是高耦合的,而消息中間件的採用,實現了鬆耦合.它基於JMS規範,支持多種傳輸協議,最重要的是它實現了異步的調用.
二、主要概念
provider(broker)
provider是消息中間件服務的提供者,在activeMQ也稱爲broker. 在兩種實現方式:第一種我們下載的activeMQ包解壓後就是一個完整的broker;第二種自己實現broker,也稱爲嵌入式的broker.
producer
producer是消息的生產者,相對於provider來說它是客戶端.它會將生產的消息發送到provider.以供consumer使用.
consumer
consumer是消息的消費者,相對於provider來說它也是客戶端,它會時刻監聽provider的對應的domain消息.一旦監聽到就會消費.
destination domain
有兩種類型:一種是point-to-point即點對點的隊列消息,一條消息只會有一個消費者;二種是publish/susribe發佈與訂閱消息,一條消息可以有多個消費者.
三、provider的配置
connectors的理解與配置
3.1 transportConnectors 消息中間件開放的協議地址,供producer和consumer訪問.
<transportConnectors>
<transportConnector name="openwire" uri="tcp://localhost:61616?trace=true" discoveryUri="multicast://default"/>
<transportConnector name="ssl" uri="ssl://localhost:61617"/>
<transportConnector name="stomp" uri="stomp://localhost:61613"/>
<transportConnector name="xmpp" uri="xmpp://localhost:61222"/>
</transportConnectors>
支持多種協議:tcp/nio/udp/ssl/openWire/ssl/stomp/xmpp/ws/mqtt/amqp/vm
3.2 networkConnectors 多個broker相互間的通信
<networkConnectors>
<!-- by default just auto discover the other brokers -->
<networkConnector name="default-nc" uri="multicast://default"/>
<!-- Example of a static configuration:
<networkConnector name="host1 and host2" uri="static://(tcp://host1:61616,tcp://host2:61616)"/>
-->
</networkConnectors>
3.2.1 靜態broker-cluster(固定已知的IP地址)
static:(uri1,uri2,uri3,...)?transportOptions
如:
<networkConnector name="host1 and host2" uri="static://(tcp://host1:61616,tcp://host2:61616)"/>
將消息發送到本地的broker會被分發到host1和host2上,producer和consumer都只連接本地的broker就做到負載均衡.
3.2.2 失敗轉移FailOver protocol
前面的負載有一個單點的問題,因此需要做一個主備的策略.
failover:(tcp://localhost:61616,tcp://remotehost:61616)?initialReconnectDelay=100
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616" updateClusterClients="true" updateClusterFilter=".*A.*,.*B.*"/>
updateClusterClients=true需要設置
結合以上兩點就能做到發的集羣效果.
3.2.3 動態broker-cluster(自動發現)
組播協議Multicast Protocol
<networkConnector name="default-nc" uri="multicast://default"/>
<transportConnector name="openwire" uri="tcp://localhost:61616?trace=true" discoveryUri="multicast://default"/>
1.需要配置discoveryUri,它是暴露的訪問接口組播發現
2.networkConnector uri=multicast://default 多broker的相互發現,使用的是默認配置相當於uri=multicast://239.255.2.3:6155?group=default
3.組播協議需要brokers在同時一個網絡.
四、消息持久化
4.1 JDBC存儲於關係型數據庫中,如mysql/oracle/Derby/PostgreSQL/SQLServer/Sybase
需要建兩張表
表ACTIVEMQ_MSGS
ID INTEGER the sequence id used to retrieve the message
CONTAINER VARCHAR(250) the destination of the message
MSGID_PROD VARCHAR(250) The id of the message producer is used
MSGID_SEQ INTEGER the producer sequence number for the message.
EXPIRATION BIGINT the time in milliseconds when the message will expire
MSG BLOB the serialized message itself
表ACTIVEMQ_ACKS
CONTAINER VARCHAR(250) the destination
SUB_DEST VARCHAR(250) the destination of the durable subscriber (could be different from the container if using wild cards)
CLIENT_ID VARCHAR(250) the clientId of the durable subscriber
SUB_NAME VARCHAR(250) The subscriber name of the durable subscriber
SELECTOR VARCHAR(250) the selector of the durable subscriber
LAST_ACKED_ID Integer the sequence id of last message received by this subscriber
4.1.1 配置-消息存儲
<persistenceAdapter>
<jdbcPersistenceAdapter dataSource="#mysql-ds"/>
</persistenceAdapter>
<bean id="mysql-ds" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver"/>
<property name="url" value="jdbc:mysql://localhost/activemq?relaxAutoCommit=true"/>
<property name="username" value="activemq"/>
<property name="password" value="activemq"/>
<property name="maxActive" value="200"/>
<property name="poolPreparedStatements" value="true"/>
</bean>
<bean id="oracle-ds" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="oracle.jdbc.driver.OracleDriver"/>
<property name="url" value="jdbc:oracle:thin:@localhost:1521:AMQDB"/>
<property name="username" value="scott"/>
<property name="password" value="tiger"/>
<property name="maxActive" value="200"/>
<property name="poolPreparedStatements" value="true"/>
</bean>
<bean id="derby-ds" class="org.apache.derby.jdbc.EmbeddedDataSource">
<property name="databaseName" value="derbydb"/>
<property name="createDatabase" value="create"/>
</bean>
<bean id="postgres-ds" class="org.postgresql.ds.PGPoolingDataSource">
<property name="serverName" value="localhost"/>
<property name="databaseName" value="activemq"/>
<property name="portNumber" value="0"/>
<property name="user" value="activemq"/>
<property name="password" value="activemq"/>
<property name="dataSourceName" value="postgres"/>
<property name="initialConnections" value="1"/>
<property name="maxConnections" value="10"/>
</bean>
4.1.2 配置-日誌存儲
<persistenceAdapter>
<journaledJDBC dataDirectory="${activemq.base}/data" dataSource="#postgres-ds"/>
</persistenceAdapter>
適用場景:對於集羣的broker用JDBC的消息和日誌存儲,可以有效的避免數據丟失.
4.2 存儲於文件中,如:kaha
AMQ message store
<persistenceAdapter>
<amqPersistenceAdapter directory="${activemq.base}/data"
syncOnWrite="true" indexPageSize="16kb" indexMaxBinSize="100" maxFileLength="10mb" />
</persistenceAdapter>
Kaha message store
<persistenceAdapter>
<kahaPersistenceAdapter directory="activemq-data" maxFileLength="10mb" />
</persistenceAdapter>
Kaha與amq message store相似,都是基於文件的存儲.相較而言,kaha的處理速度要高,但是可靠性降低.
4.3 存儲於內存中
<broker brokerName="test-broker" persistent="false" xmlns="http://activemq.apache.org/schema/core">
只需要設置persistent=false,就會直接使用內存來存儲消息.
但是需要注意設置JVM的大小和內存大小,來適應在高併發下能涉及到最大消息數據大小.
因爲它不會主要緩存消息,一旦broker宕了,是無法恢復的.
五、caching message for consumers works (緩存消費者消息)
這個主要是針對publish/subscribe的topic進行消息緩存,以達到近似的real-time.
activemq.xml的片段
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry queue=">" memoryLimit="5mb"/>
<policyEntry topic=">" memoryLimit="5mb">
<dispatchPolicy>
<strictOrderDispatchPolicy/>
</dispatchPolicy>
<subscriptionRecoveryPolicy>
<lastImageSubscriptionRecoveryPolicy/>
</subscriptionRecoveryPolicy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
subscriptionRecoveryPolicys 的類型有以下幾種:
FixedSizeSubscriptionRecoveryPolicy在內存以固定大小緩存消息,默認使用.
FixedCountSubscriptionRecoveryPolicy 在內存以固定數量緩存消息,默認使用.
QueryBasedSubscriptionRecoveryPolicy
TimedSubscriptionRecoveryPolicy 過期時間
LastImageSubscriptionRecoveryPolicy 只緩存最後一條topic消息
NoSubscriptionRecoveryPolicy 不使用緩存
可以根據不同主題設置不同的緩存策略,配置多個policyEntry,並且可以使用通配符.
六、安全訪問(securing)
6.1 簡單的xml配置用戶名和密碼
<plugins>
<simpleAuthenticationPlugin>
<users>
<authenticationUser username="admin" password="password" groups="admins,publishers,consumers"/>
<authenticationUser username="publisher" password="password" groups="publishers,consumers"/>
<authenticationUser username="consumer" password="password" groups="consumers"/>
<authenticationUser username="guest" password="password" groups="guests"/>
</users>
</simpleAuthenticationPlugin>
</plugins>
6.2 JAAS插件
http://java.sun.com/products/jaas/reference/docs/index.html
在activemq.xml中增加以下片段
<plugins>
<jaasAuthenticationPlugin configuration="activemq-domain" />
<authorizationPlugin>
<map>
<authorizationMap>
<authorizationEntries>
<authorizationEntry topic=">" read="admins" write="admins" admin="admins" />
<authorizationEntry topic="STOCKS.>" read="consumers" write="publishers" admin="publishers" />
<authorizationEntry topic="STOCKS.JAVA" read="guests" />
<authorizationEntry topic="ActiveMQ.Advisory.>" read="admins,publishers,consumers,guests" write="admins,publishers,consumers,guests" admin="admins,publishers,consumers,guests" />
</authorizationEntries>
</authorizationMap>
</map>
</authorizationPlugin>
</plugins
創建三個文件:login.config/users.properties/groups.properties
login.config
activemq {
org.apache.activemq.jaas.PropertiesLoginModule required
org.apache.activemq.jaas.properties.user="users.properties"
org.apache.activemq.jaas.properties.group="groups.properties";
};
users.properties
admin=password
publisher=password
consumer=password
guest=password
groups.properties
admins=admin
publishers=admin,publisher
consumers=admin,publisher,consumer
guests=guest
6.3 實現MessageAuthorizationPolicy接口的自定義類,來控制安全訪問.
先編譯此類,並將此包放到activemq的lib目錄下
<messageAuthorizationPolicy>
<bean class="org.apache.activemq.book.ch5.AuthorizationPolicy" xmlns="" />
</messageAuthorizationPolicy>
6.4 繼承BrokerFilter類,override相應的方法,來實現安全控制.
同時還需要一個實現BrokerPlugin接口的類.
編譯此兩個類,並放置於activemq的lib目錄下
並配置activemq.xml
<bean id="ipAuthenticationPlugin" class="org.apache.activemq.book.ch5.IPAuthenticationPlugin">
<property name="allowedIPAddresses">
<list>
<value>127.0.0.1</value>
</list>
</property>
</bean>
如spring的注入一樣配置javabean
同時需要在broker標籤中增加plugns屬性
<broker xmlns="http://activemq.org/config/1.0" brokerName="localhost" dataDirectory="${activemq.base}/data" plugins="#ipAuthenticationPlugin">
七、embedding the broker(自定義broker服務)
編程式
使用BrokerService 來配置broker,所有的參數都需要在java代碼中體現.
使用BrokerFactory來生成BrokerService,所有參數通過配置文件來實現.
<broker xmlns="http://activemq.org/config/1.0" brokerName="localhost" dataDirectory="${activemq.base}/data">
<!-- The transport connectors ActiveMQ will listen to -->
<transportConnectors>
<transportConnector name="openwire" uri="tcp://localhost:61616" /> </transportConnectors>
<plugins>
<simpleAuthenticationPlugin>
<users>
<authenticationUser username="admin" password="password" groups="admins,publishers,consumers"/>
<authenticationUser username="publisher" password="password" groups="publishers,consumers"/>
<authenticationUser username="consumer" password="password" groups="consumers"/>
<authenticationUser username="guest" password="password" groups="guests"/>
</users>
</simpleAuthenticationPlugin>
</plugins>
</broker>
xml schema引用地址:http://activemq.apache.org/xml-reference.html
spring整合
依賴包
<dependency>
<groupId>org.apache.xbean</groupId>
<artifactId>xbean-spring</artifactId>
<version>4.5</version>
</dependency>spring1.0整合
<beans>
<bean id="broker" class="org.apache.activemq.xbean.BrokerFactoryBean">
<property name="config" value="file:src/main/resources/org/apache/activemq/book/ch5/activemq-simple.xml" />
<property name="start" value="true" />
</bean>
</beans>
spring2.0整合
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:amq="http://activemq.apache.org/schema/core"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://activemq.apache.org/schema/core
http://activemq.apache.org/schema/core/activemq-core-5.14.5.xsd
">
<amq:broker brokerName="localhost" dataDirectory="${activemq.base}/data">
<amq:transportConnectors>
<amq:transportConnector name="openwire" uri="tcp://localhost:61616" />
</amq:transportConnectors>
<amq:plugins>
<amq:simpleAuthenticationPlugin>
<amq:users>
<amq:authenticationUser username="admin" password="password" groups="admins,publishers,consumers"/>
<amq:authenticationUser username="publisher" password="password" groups="publishers,consumers"/>
<amq:authenticationUser username="consumer" password="password" groups="consumers"/>
<amq:authenticationUser username="guest" password="password" groups="guests"/>
</amq:users>
</amq:simpleAuthenticationPlugin>
</amq:plugins>
</amq:broker>
</beans>八、producer的使用
九、consumer的使用
十、activeMQ優化
注:
1.本文章使用的activeMQ版本是5.1.0;maven版本是3.3.3;ant版本是1.9.9;
2.activeMQ最新版本與5.1.0有較大的差別.但核心內容是一致的.
3.ant版本依賴於JDK的版本.1.10.0的ant版本要求是JDK1.8(反正本機學習是用1.7的版本無法使用).
參考地址:http://activemq.apache.org/
參考文檔:activeMQ in action