Further, while all these tricks are amazing, they are not currently of much use to fungible data scientists at FAANG and in the greater Fortune 500 because they haven't been added to any of the major frameworks in a manner that is braindead/automagic. Further, most of these are but one GPU generation away from general availability if they become popular like FP16/32 mixed precision did.
That said, the acquisition of DeePhi by Xilinx ought to be a sufficiently strong signal to add them to said frameworks, no?
Finally, my gut instincts (based on the monotonically decreasing speech recognition quality of my increasingly powerful Android phones over time, and the bizarre spelling errors it now makes such as "tech kneeq" as one example) is that one cannot rate the quality of these techniques from a single metric, and it seems like the decision to foist them on users may be based on one. #ItsAdversariesAllTheWayDown
I spent a little bit of time trying to understand exactly what DeePhi provides, but couldn't find any whitepapers or anything. Do you know any documentation on which of these approximations are supported by DeePhi?
I don't know which of these techniques they actually built, my experience with these techniques is several encounters with salespeople from multiple vendors who present them running on FPGAs and then compare performance to non-approximated versions of the same network on a GPU to show the amazing speed gains with minimal reduction in test set metrics. This is, of course, not the whole story.
My response to them is to tell them to implement and compare against the same tricks on a GPU if you expect me to believe the throughput performance delta. I usually get some hand-waving about how that's not possible or pointless yadda yadda yadda and the conversation goes downhill from there. That doesn't mean I don't like these tricks or that I don't think they're useful, it means that based on the scars from a very long industry career I don't buy snake oil.
Further, while all these tricks are amazing, they are not currently of much use to fungible data scientists at FAANG and in the greater Fortune 500 because they haven't been added to any of the major frameworks in a manner that is braindead/automagic. Further, most of these are but one GPU generation away from general availability if they become popular like FP16/32 mixed precision did.
That said, the acquisition of DeePhi by Xilinx ought to be a sufficiently strong signal to add them to said frameworks, no?
https://www.xilinx.com/news/press/2018/xilinx-announces-the-...
Finally, my gut instincts (based on the monotonically decreasing speech recognition quality of my increasingly powerful Android phones over time, and the bizarre spelling errors it now makes such as "tech kneeq" as one example) is that one cannot rate the quality of these techniques from a single metric, and it seems like the decision to foist them on users may be based on one. #ItsAdversariesAllTheWayDown