We are excited to announce today that we have scaled our drug design platform to 10,000+ cores on Amazon Web Services’ Elastic Cloud Compute (AWS EC2) utilizing spot instances. Spot instances are regular EC2 instances that are run on unutilized cloud hardware, allowing AWS to recoup the cost of idle hardware. The spot instances are acquired through an auction. Because the price can fluctuate, due to demand, spot instances may spin up and down at any time. The Numerate drug design platform and the Numatix distributed framework on which it runs were both designed from the ground up to handle such ephemeralness with ease, allowing us to take full advantage of the cost savings of spot instances, which can be as low as a third of the on-demand price.
Numerate’s Drug Design Platform Scales to 10,000+ Cores Using Spot Instances;
Large-Scale Data Analyses Enabled by Numatix, Numerate’s Distributed Framework
SAN BRUNO, CA – June 10, 2012 – Numerate, Inc., a technology platform company that is leveraging proprietary algorithms and the power of cloud computing to transform the drug design process, announced today that its drug discovery platform, based on its open-source distributed framework, Numatix, has been shown to reliably and cost-effectively scale to 10,000+ cores using spot instances on Amazon Web Services’ Elastic Cloud Compute (AWS EC2). The 10,000+ cores were used to screen virtual compounds against predictive assay models in one of the company’s commercial partnerships.
Numatix is a dataflow processing platform developed by Numerate that enables great scalability and flexibility in distributed computing with minimal operational overhead. Combining online processing with detailed dataflows, Numatix allows for the deep and interactive analyses of large data sets. At Numerate, Numatix is used for the computational assessment of billions of molecules in the search for new drugs and for the development of complex systems biology models of drug behavior in various animals and humans.
“This level of scalability is essential for Numerate due to growing demand for our drug design services in the pharmaceutical and biotechnology industry,” said Nigel Duffy, Ph.D., Chief Technology Officer. “This achievement demonstrates the power of the Numatix platform and its ability to tackle large-scale computational problems cost effectively. We designed Numatix from the ground up to be extremely robust to all types of failures, allowing us to take full advantage of spot instances and their cost savings.”
“We were able to scale up to 10,000 nodes in the US-East zone of EC2,” stated Brandon Allgood, Ph.D., Director of Computational Science. “And because the Numatix platform uses spot instances, the cost was only 2.7¢ per core-hour, or $270 per hour for a 10,000-core cluster – just one-third of the on-demand cost. Moreover, we were able to reach this scale while handling spot instance ephemeralness and maintaining a high level of security.”
Allgood added, “The limiting factor on this run was our cloud provider-independent security layer. On less sensitive types of workloads, we expect that Numatix should be able to scale well beyond 10,000 cores.”