If you can upload a PDF and have it served on the root domain, eg, gmail.com, then you can do session hijacking and other XSS. It’s actually pretty bad. XSS used to be thought of as “not that bad”, but today it is considered pretty bad.
Yes, I'm not here to downplay the severity of XSS. Rather, I'm trying to be specific about the potential attack vector here.
If you're just viewing a PDF using the built-in pdf.js in Firefox, then (AFAIK) it doesn't matter what site you downloaded it from, because pdf.js isn't running in the context of the website, so it doesn't have access to that site's locally-stored data (including cookies). Instead it's running in the origin mentioned above, with the accompanying concerns.
So the XSS (again, as far as a web browser is concerned) would be if the site itself is shipping pdf.js for viewing PDFs inside the webpage itself. As you suggest, Gmail lets you preview PDFs, so XSS would be a concern there, but only if Gmail is using pdf.js.