Smoking weed in your home is already all but legal in Sweden. In the super unlikely case you get caught you pay a small fine and is free to go, the people I know who smoke weed aren't worried about the Police, more annoyed than anything. And regular tobacco smoking has mostly disappeared since it has became so inconvenient.
You mentioned tobacco, which is recently saw laws made against it in Sweden.
I then also take the approach in what laws should then be remove in order not to unfairly "discriminate against certain behavior" (which is sadly so common nowadays).
Now what's so annoying about some person taking a line of coke in public? (do you get the drift?)