Show Contents List
You are here:

Benchmark Configuration

The following describes the benchmark configuration, unless otherwise noted below.

• AMI : LANSA Web Server 20150216-2103 (ami-992452a3).
• Windows Server 2012 R2 with all Windows updates applied.
• Multi AZ Auto Scaling Group .
• SQL Server Web 11.00.2100.60.v1, Single AZ.
• LANSA performance improvements in V14. 
• Tested 12th March 2015 with a beta version of Visual LANSA V14. 
• AWS Region ap-southeast-2 (Sydney), unless otherwise stated.
• Load test tool Loadster Workbench 3.5.6 running in 5 Regions - Virginia, Singapore, Ireland, Brazil, Sydney.
• Benchmark includes select, insert, update, and delete transactions with appropriate delays between transactions for each user.
• The Base Cost column provides indicative pricing. Only the EC2 instances and the RDS instance are included in the cost. There are other charges that are generally a small percentage of these costs which you will incur. You should monitor the costs closely to ensure it is within your budgetary constraints. RDS costs do not include data storage costs and, in particular, neither Provisioned IOPS increased charges. Autoscaling may also increase the number of web servers running to handle an increased load, further increasing the costs.

  Concurrent Users	Web Servers (# EC2 Instances)	EC2 Instance Type (Max CPU %)	RDS Instance Type	SQL Server	Avg Response Time (seconds)	Max Response Time (Seconds)	Test #	Base Cost ($ per hour)
  30	4	m3.medium (67%)	db.m3.medium	sqlserver-web	1.72	10.6	188	0.73
  30	2	t2.micro (67%)	db.m3.medium	sqlserver-web	1.3	5.82	212	0.246
  40	4	t2.micro (47%)	db.m3.medium	sqlserver-web	1.56	6.65	214	0.282
  40	4	t2.small (47%)	db.m3.medium	sqlserver-web	1.58	6.81	285	0.354
  80 (Single AZ)	10	m3.medium (75%)	db.m3.large	sqlserver-web	1.45	7.66	166b	1.72
  80	10	m3.medium (77%)	db.m3.large	sqlserver-web	1.75  (20% slower than single-AZ)	6.86	185	1.72
  80	5	t2.micro (79%)	db.m3.large	sqlserver-web	1.45	5.59	221	0.51
  80	3	t2.medium (67%)	db.m3.large	sqlserver-web	1.5	122.11	226	0.636
  80	4	m3.large (65%)	db.m3.large	sqlserver-web	1.15	4.93	238	1.456
  80 (Single AZ)	10 (Virginia)	m3.medium (85%)	db.m3.large	sqlserver-web	1.34	14.67	197	1.72
  80 (Single AZ)	10 (Sao Paulo)	m3.medium (78%)	db.m3.large	sqlserver-web	1.1	3.9	206	1.72
  80 (Single AZ)	10 (Tokyo)	m3.medium (60%)	db.m3.large	sqlserver-web	1.3	6.16	207	1.72
  100	8	m3.medium (90%)	db.m3.xlarge	sqlserver-web	1.9	11.47	151	1.88
  600	60	m3.medium (77%)	db.m3.2xlarge	sqlserver-web	1.85	7.75	160b	9.48
  100	10	m3.medium (80%)	db.r3.xlarge	sqlserver-web	1.5	7.2	158	2.18
  300	30	m3.medium (70%)	db.r3.xlarge	sqlserver-web	2.1	7.59	164	4.78
  300	10	t2.medium (55%)	db.r3.xlarge	sqlserver-web	2.05	7.08	233	1.6
  300	10 (PIOPS 2000)	t2.medium (58%)	db.r3.xlarge	sqlserver-web	1.84 (10% improvement)	7.02	261	1.6
  500	60	m3.medium (73%)	db.r3.2xlarge	sqlserver-se	1.65	7.5	175	10.82
  500	15	t2.medium (75%)	db.r3.2xlarge	sqlserver-se	1.1	5.48	232	4.1
  500	10	m3.xlarge (82%)	db.r3.2xlarge	sqlserver-se	1.2	5.08	242	8.2
  500	90	m3.medium (56%)	db.r3.2xlarge	sqlserver-se	1.5	7.63	177	14.72
  600	60	m3.medium (72%)	db.r3.2xlarge	sqlserver-se	2	8.24	173	10.82
  750	90	m3.medium (56%)	db.r3.2xlarge	sqlserver-se	3.25	14.32	170	14.72
  750	18	t2.medium (68%)	db.r3.2xlarge	sqlserver-se	2.3	9.02	236	4.316
  750	18  (PIOPS 1000)	t2.medium (73%)	db.r3.2xlarge	sqlserver-web	2.1 (10% improvement)	9.28	266	3.096
  900	120	m3.medium  (38%)	db.r3.2xlarge	sqlserver-se	4.75	21.57	167	18.62
  900	30	t2.medium (40%)	db.r3.2xlarge	sqlserver-se	3.5	11.67	234	5.18
  900	20	m3.xlarge (37%)	db.r3.2xlarge	sqlserver-se	3.4	12.29	241	13.38
  900	10	m3.2xlarge (38%)	db.r3.2xlarge	sqlserver-se	3.45	19.47	245	13.38
  900	10	c3.2xlarge (31%)	db.r3.2xlarge	sqlserver-se	3.45	15.98	246	10.54
  900	10	c4.2xlarge (25%)	db.r3.2xlarge	sqlserver-se	3.45	17.41	247	10.75

   |
  Note that little difference was seen between db.m3.large and db.m3.xlarge for the benchmark. So this would indicate that it's not worth spending extra for db.m3.xlarge. It is unclear as to why this is so and why moving from db.m3.large to db.m3.2xlarge, which is 4 times the price yet gives 6 times the performance. If the instance types are compared, apart from doubling CPU and memory for each step, db.m3.2xlarge has High network performance. So it seems that this is the factor which allows the dramatic improvement in throughput for db.m3.2xlarge. Of course this comment is purely in the context of this benchmark. Another application may experience different behavior.
  Show Contents List