環境
CentOS 7.2
JMeter 3.2
Topologie
- (Client|Master): the system running Jmeter which controls the slave remote (and doesn't send any request)
- (Server|Slave|Remote) Node: the system running JMeter in server mod (jmeter-server), which takes commands from the client and send the test plan requests to the target system(s)
- Target: the server to stress test
Configure Client Node
JMeter uses Java RMI [Remote Method Invocation] to interact with objects in a distributed network.
JMeter master and slave communicate as shown in the below picture.
remote_hosts: in client JMeter machine,add value of JMeter servers IP address. Multiple servers can be added comma-delimited. A more convenient way to specify remote host is from command line. Use -R command line to specify which remote host to use
client.rmi.localport: Specify client.rmi.localport in jmeter.properties file (for example - client.rmi.localport=25000). This is the port on which the local JMeter instance will listen for the Remote Method Invocation (i.e. RMI connections) from the slave nodes. Results would be sent to client from slave nodes on this port
Configure Server Nodes
Server nodes should be running same version of JMeter and java as client
Test datafiles are not sent across by client, hence they should be available in appropriate directory in each server. You can keep test data file in /bin folder in each JMeter server. It gets you rid of specifying full qualified path of test data file on each server instance.
To use different values for properties use corresponding values for user.properties and system.properties on each server
- server.rmi.localport: By default, RMI uses a dynamic port for the JMeter server engine. This may cause issues with firewalls, so with versions of JMeter after 2.3.2 you can define the JMeter property server.rmi.localport to control this port number. To use a specific port for the JMeter server engine, define server.rmi.localport property before starting the server
JMeter/RMI requires:
a connection from the client to the server. Default port 1099.
and a reverse connection in order to return results from the server to the client. This will use a high-numbered port controlled by jmeter property “client.rmi.localport”
Prerequisites
All the nodes (client and servers):
- are running the same version of JMeter.
- are running the same version of Java. For JMeter 2.9, Java 6 runtime or higher
- have extra test data files available in the appropriate directory
are on the same subnet - can communicate (the firewalls must be turned off of set up to allow the connections)
- must be started with the same property (file and command line option)
JMeter安裝
下載解壓即可。
wget https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-3.2.tgz
tar -xzf apache-jmeter-3.2.tgz
JMeter命令
-h 幫助 -> 打印出有用的信息并退出
-n 非 GUI 模式 -> 在非 GUI 模式下運行 JMeter
-t 測試文件 -> 要運行的 JMeter 測試腳本文件
-l 日志文件 -> 記錄結果的文件
-r 遠程執行 -> 啟動遠程服務
-H 代理主機 -> 設置 JMeter 使用的代理主機
-P 代理端口 -> 設置 JMeter 使用的代理主機的端口號
$ jmeter -n -t test1.jmx -l logfile1.jtl -H 192.168.1.1 -P 8080
方案1
client,server均采用手動部署方式，方便調試查錯。
啟動server：
[root@VM_1_111_centos bin]# ./jmeter-server -Djava.rmi.server.hostname=10.0.1.111
Created remote object: UnicastServerRef [liveRef: [endpoint:[10.0.1.111:36522](local),objID:[-77441bd0:15bf138f424:-7fff, -7104802513269301355]]]
啟動client：
[root@VM_1_246_centos bin]# ./jmeter -n -t baidu-test.jmx -R10.0.1.111:31099 -Djava.rmi.server.hostname=10.0.1.246
Creating summariser <summary>
Created the tree successfully using baidu-test.jmx
Configuring remote engine: 10.0.1.111:31099
Starting remote engines
Starting the test @ Wed May 10 15:19:58 CST 2017 (1494400798945)
Remote engines have been started
Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
summary + 1 in 00:00:01 = 1.9/s Avg: 374 Min: 374 Max: 374 Err: 0 (0.00%) Active: 1 Started: 1 Finished: 0
summary = 1 in 00:00:01 = 1.9/s Avg: 374 Min: 374 Max: 374 Err: 0 (0.00%)
Tidying up remote @ Wed May 10 15:20:00 CST 2017 (1494400800718)
... end of run
可以看到server上的調用信息：
Starting the test on host 10.0.1.111:31099 @ Wed May 10 15:19:59 CST 2017 (1494400799839)
Finished the test on host 10.0.1.111:31099 @ Wed May 10 15:20:00 CST 2017 (1494400800740)
方案2
壓測需要部署大量server節點，采用docker部署。
生成JMeter base鏡像需要的Dockerfile如下：
# Use Ubuntu
FROM ubuntu
MAINTAINER adeng <xxxxxx@qq.com>
# Install wger & JRE
RUN apt-get clean && \
apt-get update && \
apt-get -qy install \
wget \
default-jre-headless \
telnet \
iputils-ping \
unzip
# Install jmeter
RUN mkdir /jmeter \
&& cd /jmeter/ \
&& wget https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-3.2.tgz \
&& tar -xzf apache-jmeter-3.2.tgz \
&& rm apache-jmeter-3.2.tgz \
&& mkdir /jmeter-plugins \
&& cd /jmeter-plugins/ \
&& wget https://jmeter-plugins.org/downloads/file/JMeterPlugins-ExtrasLibs-1.4.0.zip \
&& unzip -o JMeterPlugins-ExtrasLibs-1.4.0.zip -d /jmeter/apache-jmeter-3.2/
# Set Jmeter Home
ENV JMETER_HOME /jmeter/apache-jmeter-3.2/
# Add Jmeter to the Path
ENV PATH $JMETER_HOME/bin:$PATH
生成JMeter slave鏡像需要的Dockerfile如下：
FROM malfurionpd/jmeter-base
MAINTAINER adeng <xxxxxx@qq.com>
# Ports to be exposed from the container for JMeter Slaves/Server
EXPOSE 31099 32500
# Application to run on starting the container
ENTRYPOINT $JMETER_HOME/bin/jmeter-server \
-Dserver.rmi.localport=32500 \
-Dserver_port=31099
可以參考這里在Dockerhub上生成鏡像，或者本地生成：
docker build -t malfurionpd/jmeter-slave .
部署三個容器，主機端口直接映射容器expose的端口，client直接遠程調用：
[root@VM_1_246_centos bin]# ./jmeter -n -t baidu-test.jmx -R10.0.2.104:31099,10.0.3.248:31099,10.0.3.180:31099 -Djava.rmi.server.hostname=10.0.1.246
Creating summariser <summary>
Created the tree successfully using baidu-test.jmx
Configuring remote engine: 10.0.2.104:31099
Configuring remote engine: 10.0.3.248:31099
Configuring remote engine: 10.0.3.180:31099
Starting remote engines
Starting the test @ Wed May 10 17:09:50 CST 2017 (1494407390007)
Remote engines have been started
Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
summary = 2 in 00:00:01 = 3.7/s Avg: 174 Min: 112 Max: 236 Err: 0 (0.00%)
Tidying up remote @ Wed May 10 17:09:51 CST 2017 (1494407391402)
summary + 1 in 00:00:01 = 0.8/s Avg: 721 Min: 721 Max: 721 Err: 0 (0.00%) Active: 0 Started: 2 Finished: 3
summary = 3 in 00:00:02 = 1.7/s Avg: 356 Min: 112 Max: 721 Err: 0 (0.00%)
Tidying up remote @ Wed May 10 17:09:52 CST 2017 (1494407392649)
... end of run
... end of run
方案3
參考這里，使用Rancher部署測試集群。
坑1
直接執行jmeter-server報錯：
[root@localhost bin]# ./jmeter-server
Created remote object: UnicastServerRef [liveRef: [endpoint:[127.0.0.1:39150](local),objID:[-3126fe29:14300b1102e:-7fff, -4078314045196249121]]]
Server failed to start: java.rmi.RemoteException: Cannot start. localhost is a loopback address.
An error occurred: Cannot start. localhost is a loopback address.
解決方法如下:
In latest version, you can run your script with:指定本地IP
./jmeter-server -Djava.rmi.server.hostname=xxx.xxx.xxx.xxx
坑2
使用全容器部署時，不指定Host IP情況下 RMI總是出問題:
root@VM_7_2_centos:/jmeter/apache-jmeter-3.2/bin# ./jmeter -n -t baidu-test.jmx -Djava.rmi.server.hostname=10.0.7.2 -Dclient.rmi.localport=60000 -R10.0.7.14,10.0.7.7
Creating summariser <summary>
Created the tree successfully using baidu-test.jmx
Configuring remote engine: 10.0.7.14
Configuring remote engine: 10.0.7.7
Starting remote engines
Starting the test @ Tue Jun 13 12:59:01 UTC 2017 (1497358741786)
Error in rconfigure() method java.rmi.ConnectIOException: Exception creating connection to: 10.42.37.96; nested exception is:
java.net.NoRouteToHostException: No route to host (Host unreachable)
Error in rconfigure() method java.rmi.ConnectIOException: Exception creating connection to: 10.42.255.116; nested exception is:
java.net.NoRouteToHostException: No route to host (Host unreachable)
Remote engines have been started
Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
解決方法和上邊一樣，必須想辦法指定 ./jmeter-server -Djava.rmi.server.hostname=xxx.xxx.xxx.xxx 。
參考文章1
參考文章2
參考文章3
參考文章4
參考文章5
參考Docker三部曲之1
參考Docker三部曲之2
參考Docker三部曲之3
