This paper presents our efforts in addressing these issues by combining an efficient use of hard disk with the distribution of the state space on several computing nodes.
This allows a multi-sample experiment to be automatically distributed among several computing nodes (the steps depicted in Figure 1 are run in parallel for each sample).
A distributed system [ 3] is a setup in which several independent computers (computing nodes) participate in solving the problem of processing a large volume of and variety of structured/semi-structured/unstructured data.
These runs took several months even leveraging the thousands of compute nodes available in Blue Gene.
Next, we describe several standards for efficient communication between CPUs within a single board compute node, followed by a description of several switch fabric architectures used to interconnect these compute nodes to form large parallel machines.
Computing an evolution as shown in Additional File 1 takes several hours on the 32 to 64 Infiniband-connected compute nodes we have typically employed, and yields many gigabytes of data.
Files in the GFS are automatically partitioned into large chunks (64 MB by default), which are replicated to several physical disks (three by default) attached to the compute nodes.
GPUs offer several times the floating point performance and memory bandwidth of current standard two socket CPU compute nodes, e.g. NVIDIA C2070 vs. Intel Xeon Westmere X5650.
The computations are run on a Linux cluster computing environment with 200 compute nodes, and each node takes around 36 hours.
Nowadays, novel architectures like Cloud and Internet of Things (IoT) make available several resources like computing nodes, environmental sensors etc. that enable the introduction of more and more intelligent systems able to face complex situations.
