-p precision does not mean digits?

[johnblommers]

Consider the following code which should generate a million digits of pi:

val -p 1000000 -e "pi" | wc -c
301032

Instead val is only giving up about 301032 characters, which includes the leading 3, the decimal point and the endline character.

[gahooa]

(copied from my reddit post)

There are some odd precision issues at high precision numbers.

try:

$ val -p 65473 -e '(2^(2^(2^(2^(2)))))'
>>> last digits are: 7777506072339445587895905719156736

vs

$ val -p 65472 -e '(2^(2^(2^(2^(2)))))'
>>> last digits are: 7777506072339440000000000000000000

Note: the resulting numbers correctly have 19729 digits in either case. However, I noticed that I have to specify -p 65473 in order to see all the digits.

Why would that be?

[gahooa]

Consider the following code which should generate a million digits of pi:

val -p 1000000 -e "pi" | wc -c
301032

Instead val is only giving up about 301032 characters, which includes the leading 3, the decimal point and the endline character.

I am starting to wonder if -p is representing binary precision.

301032 * log2(10)
1000006

[terror]

Thanks @gahooa, -p or --precision does indeed represent binary precision. This gets passed into all floating point computations. I could add a --fix or --digits option to specify the number of decimal places one would like to see in the output, if that's useful.

[johnblommers]

I could add a --fix or --digits option to specify the number of decimal places one would like to see in the output, if that's useful.

IMHO adding the option --digits would be very useful. For the bit-oriented users --precision would still be useful.

Consider clarifying the val --help output description for --precision to mean bits not digits.

BTW it is delightful how quickly val is able to compute a million digits of Pi on my small form factor PC with its lowly AMD Ryzen 5 3550H CPU

[gahooa]

I agree with @johnblommers that it would be helpful to have it in the docs as well as both decimal and binary precision options.