Cash exists and can change hands without there being a record of transactions. At the lower end of the economy, many people are paid in cash and transact in cash, and as you've said, society hasn't imploded.

The government tends to go after lower and middle class people when enforcing laws. I think the benefits of preserving privacy outweigh the IRS collecting the $600 someone who makes $30k owes.