To the best of our knowledge, this is the first systematic work in the consumer domain aimed at the definition of a large lexicon, construction of a large benchmark data set, and annotation of videos in a rigorous fashion.
Based on the literature, we defined five building blocks of shared savings models that encompass the definition of the scope of the program, the calculation of health care expenditures, the construction of a savings benchmark, the assessment of savings and the rules and conditions under which savings are shared.
In general, our comparison of the prediction results from different benchmarks demonstrated that the assessed performance of a method depends on the actual construction of the benchmark set.
For more details on construction of the benchmark, see Nawrocki and Eddy (2007).
In order to account for as many effects as possible in the construction of this benchmarking study, a literature review was conducted, reviewing past benchmarking studies in the areas of psychology and risk assessment.
Whereas a theoretical analysis is typically a mathematical statement about the method, an empirical analysis of an algorithm typically involves its computer implementation, and the construction of benchmark data sets on which to evaluate the algorithm's performance.
The results on benchmark graphs led to the abstract construction of a worst-case input graph, which was a consequence of identifying The Problem of Ties.
The scope of the cooperative effort includes the sharing and interoperability of tools, the alignment of theory and practice, the identification of challenge problems, the construction of benchmark suites, and the execution of large-scale experiments.
There was little in the way of a benchmark.
In the absence of a benchmark we set out to create a benchmark," explains Brown.
What is the moral status of a benchmark?
