There is a lot of engineering work to do before Dremio is going to handle the highest performance workloads.
"The benefit of the unified platform is as companies and cloud providers are building out their infrastructure, they can standardize on a single unified architecture that supports the entire range of high-performance workloads.
Kurtzer now wants to bring Singularity to the enterprise with a focus not just on the HPC commercial market, but other high performance computing workloads such as artificial intelligence, machine learning, deep learning and advanced analytics.
As IBM's CTO for Watson and Cloud Bryson Koehler told me, this now also opens up the ability to attach GPUs to these machines, which in turn enables the kind of machine learning and high performance computing workloads that many enterprises are now starting to experiment with.
Given the presence of the Nvidia GPUs, it's no surprise that the system is meant to be used for machine learning and deep learning applications, as well as the usual high performance computing workloads for research in energy and advanced materials that you would expect to happen at Oak Ridge.
For commercial applications, KVM and Xen work well but High Performance Computing (HPC) workloads require more CPU cycles during migration process that may not be fulfilled with current KVM rate control and target downtime heuristics, which leads to service degradation drastically.
Furthermore, we highlight the difference between High Performance Computing (HPC) workload and Internet-based services workload.
To do so, Jackson quantitatively examines the performance of a set of benchmarks designed to represent a typical High Performance Computing (HPC) workload running on the Amazon EC2 platform.
By using object storage as an enabling layer, complemented by native file system and VM storage access, they can not only deliver storage at massive scale, but also provide high performance for a variety of workloads.
"These products help customers to optimize and centralize their high performance computing (HPC) and visualization workloads while also providing tools that are a great fit for distributed workforces making use of mobile devices," AWS's Jeff Barr wrote in a blog post announcing the deal.
