precision
Sign in to saveAlso known as artithmetic precision, significance
measure of random errors in a system: the variation among measurements of the same value
Connections
floating point
Entity
significant figure
Entity
computer science
Entity
bit
Entity
machine learning
Entity
computer architecture
Entity
integer data type
Entity
word
Entity
computation
Entity
IEEE 754
Entity
double-precision floating-point format
Entity
truncation
Entity
single-precision floating-point format
Entity
arbitrary-precision arithmetic
Entity
round-off error
Entity
Granularity
Entity
half-precision floating-point format
Entity
quadruple-precision floating-point format
Entity
extended precision
Entity
minifloat
Entity