Sentence examples for precision in the computation from inspiring English sources

For adequate precision in the computation of the response surface, we thus need to first search the DoE for such structures.

The FPGA implementation provides higher frame rates and less power consumption, whereas the GPU implementation provides more precision in the computation and therefore higher quality output images.

The clusters are not numbered in the exact order of significance due to the limits of numerical precision in the computation of p-values.

When the denominator is zero, the ratio should actually be 0/0 but can be a large value subject to precision in the numerical computations.

It is shown that the precision in the Added Masses computation and the computational time depend on the meshing quality of the model and power of the PC on which the software runs.

Additionally, the algorithm calculates the precision of the computation in a straightforward way.

Double precision was used in the computations, and solutions were considered converged when the normalized sum of the absolute dimensionless residuals of the discretized equations was less than 10−6.

We used 0.02 as the lowest allele frequency used in the computation to balance bias and precision [ 156], including a parametric 95% C.I. in the random mating model.

The results obtained show that including SNP information in the computation of the relationship matrix between individuals can increase the precision of the genetic parameter estimates up to twice the precision obtained by using pedigree information only [ 51].

In some test cases, all particles end up having zero weights due to finite precision in computation.

The checking confirmed the adequate precision of the computations.