random module gives varying output for different Python versions

[tcld]

As stated here #138 (comment) there seems to be a change in the random number generator between Python 2 and Python 3.

Starting from the same seed, the outputs between the two versions differ.

Python 3.4.3 (default, Oct 14 2015, 20:28:29) 
[GCC 4.8.4] on linux
>>> import random
>>> random.seed(0)
>>> for x in range(0, 10):
...     random.randint(0, 1000000)
885440
403958
794772
933488
441001
42450
271493
536110
509532
424604

Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import random
>>> random.seed(0)
>>> for x in range(0, 10):
...     random.randint(0, 1000000)
844422
757955
420572
258917
511275
404934
783799
303313
476597
583382

This might be due to the different versions of gcc that have been used to compile the Python versions (I don't know how to test that right now) but since the maps here #138 (comment) are extremely similar I would expect that platec isn't influenced by the whole problem.

EDIT Here the pure code, if you want to quickly redo this:

import random
random.seed(0)
for x in range(0, 10):
    random.randint(0, 1000000)

import random
random.seed(0, version = 2)
for x in range(0, 10):
    random.randint(0, 1000000)

[ftomassetti]

Let's start looking for alternatives:

• Simple Mersenne-Twister in Python http://code.activestate.com/recipes/578056-mersenne-twister/

[tcld]

There is a problem left: I looked at the implementation of the noise module, and that one makes "hardcoded" use of random, too.

[tcld]

Overall I think this is not a big problem, just one that should be fixed. I wanted to flesh out this #133 anyway to find a completely deterministic way of generating maps, even if the order of the steps differed. So I will do this all in one swoop, then. Might be done within a couple hours or a couple days, I don't know yet how much time I can put into it.

[ftomassetti]

Ok, but that is used during world generation, right? While our tests use pre-generated worlds and then just generate maps from those worlds, correct?
Anyway it is almost impossible to get the same worlds given the same seed on different platforms because of the C++ code of plate-tectonics. We tried by adopring a spefic library for random generation, to use fixed size types (an int or a long can have different sizes on different architectures) but still the results are not the same because floating point operations produce slightly different results depending on the optimization done by the compiler. So basically we gave up on that.
We should just get the same maps given the same world and for that we should use just python code and I was thinking the noise module was not involved in that phase.

[ftomassetti]

(closed by mistake, sorry)

[tcld]

Right, noise is only used during generation (I think). But referring to my last message, it is still kind of a problem since there is no way to replace the RNG that is used by the noise-module.

You are right. We can most likely fix the Python side. As for C++, did you try implementing your own RNG?

[ftomassetti]

Yes

On Wed, Oct 21, 2015 at 8:53 AM, tcld notifications@github.com wrote:

Right, noise is only used during generation (I think). But referring to my
last message, it is still kind of a problem since there is no way to
replace the RNG that is used by the noise-module.

You are right. We can most likely fix the Python side. As for C++, did you
try implementing your own RNG?

—
Reply to this email directly or view it on GitHub
#145 (comment)
.

Website at http://tomassetti.me http://www.federico-tomassetti.it
GitHub https://github.com/ftomassetti

[tcld]

Oh, well. Then let's forget about that part permanently.^^

[tcld]

I just noticed this: https://hg.python.org/cpython/file/3.4/Lib/random.py

* The random() method is implemented in C, executes in a single Python step,

  and is, therefore, threadsafe.

This might actually be difficult without getting some kind of high-level RNG going. There is probably something like that available somewhere, but I guess it is going to be quite slow.

[psi29a]

Python's default RNG is supposed to be predictable and uses Mersenne-Twister by default.

Keep in mind that the rng is seeded by time by default when imported. So if we want to be predictable, we need to seed the rng with a timestamp and keep this timestamp in our world format.

[tcld]

Read the first post in here. The problem is that the seed does not make for predictable output.

[psi29a]

And if you did the same thing twice in the same Python environment?