There are 200k federal prisoners out of about 1600k total. The fact that federal numbers are 50% drugs doesn't change the fact that the overall numbers.

If we let out our non-violent criminals, then the prison population will only be cut by half. The majority of people in prison are violent criminals.