This is in continuation with the last benchmark we performed in one of our previous posts. When I submitted the last article on the PostgreSQL reddit, a couple of users wanted to know how PostgreSQL performed on the Amazon RDS for PostgreSQL that was launched recently. We were also curious so we did this benchmark. Rackspace also overhauled their cloud infrastructure and launched a new breed of performance servers with SSD disks recently. We wanted to test these too.
As in previous benchmark, in this benchmark we created 2 servers, one DB server and other benchmark server.
This was a db.m1.xlarge instance with 15 GB of RAM. The disk was a 400GB of EBS volume with 4000 provisioned IOPS. (Amazon required the storage to IOPS ratio in the range 1:3 to 1:10) We selected 9.3.1 version of the PostgreSQL, which is the only option as of now. We didn’t use the multi A-Z option that Amazon recommends for production deployments. This was done because all other benchmarks didn’t use replication.
This was used to run pgbench. This was same as used in the previous benchmark. An m1.xlarge instance with 15 GB RAM and default 8GB EBS volume without provisioned IOPS. We used the Amazon AMI and installed pgbench using yum.
For Rackspace benchmark we created 2 servers with their Performance 2 Flavor Class each with 15GB of RAM. These servers are equipped with SSD.
This ran Ubuntu 12.04. The version of PostgreSQL was 9.1 and was installed using apt-get. Here we could have used PostgreSQL 9.3. But we wanted to keep these numbers relevant to the previous benchmark we did which also used PostgreSQL 9.1 from the standard Ubuntu repository.
We used Cent OS 6.4 for this and pgbench was installed using yum.
Monthly Pricing For The Database Servers
Around $945 per month
Around $500 per month
First we ran the pgbench command with a scale of 100, which generates 10 million rows in the accounts table. The size of database thus created was around 1.5GB.
For Rackspace, after setting up the server we changed the postgresql.conf for max connections upto 400. Also we modified /etc/sysctl.conf and set the kernel.shmmax value to around 7GB.
We ran the benchmark 2 times on each of the servers. In the first run (henceforth referred as ‘without optimization’) we used the default values for ‘shared_buffers’ and ‘effective_cache_size’ configuration values. In the second run (henceforth referred as ‘with optimization’) we changed ‘shared_buffers’ to 4GB and ‘effective_cache_size’ to 8GB.
For Amazon RDS no changes were made to the configuration. We just used the configuration as provided by Amazon. So for RDS there is only one set of configurations.
For each run we did 4 sub tests with 96, 128, 256 and 392 clients in read-write (RW), read only (R), Write only (W) modes. Each of these tests were run for 10 minutes (600 seconds.)
As said in the last blog post, these are apples to oranges comparison. Then one would ask why do them? The main reason is about evaluating choices. Many of our clients want to deploy their applications in the cloud. Right now Amazon is the leader in the Cloud based deployments. However many new companies have entered in this market. Each cloud provider has very different architecture and hardware. Right now AWS has definitely one of the most flexible and advanced architectures than most of the cloud providers who are providing solutions based on OpenStack. Given this we need to determine which cloud will give better results for the application in question. There are many parameters for selection of a right cloud for your application. Some of them are cost, ability to scale, stability and uptime.
In a web application, if one uses shared nothing architecture, it is quite easy to scale at application server level. But in this the state gets pushed to the database and database becomes the main resource that needs to scale vertically. Of course one could use sharding but that requires quite a few changes to the data access layer of the application.
This is why I wanted to see how long I could go on with these cloud servers with a reasonably powerful configuration before we start implementing sharding.
We have presented the results above. One thing I would like to note that for RDS now Amazon supports up to 30,000 provisioned IOPS volumes. But that pushes the cost of the instance above $4000 per month.
I hope these tests help you in some way while making the choice for your next cloud deployment. Thanks for reading.