The argument isn't saying that arithmetic is random. It's saying that a lot of properties of number systems behave as if they are random.

A good example is finding arithmetic sequences of length K in the prime numbers (for example, the sequence 3, 5, 7 is an arithmetic sequence of length 3, as is 17, 23, 29). It can be shown that random sets that have a density similar to the primes (i.e. the chance that N is in the set is proportional to 1 / log(N)) have arbitrarily long arithmetic sequences - as, in fact, do the prime numbers.

"a lot of properties of number systems behave as if they are random"

Choosing a weird base (and yes, base ten is weird) certainly contributes to this effect.

This looks like an interesting perspective. Do you have any pointers to discussions of topics that become more intuitive/clearer when thought about in other bases ?

The whole topic of "why does this random (in base 10) number suddently behaves strange" doesn't exist in e.g. base 2.