OpenAI Computer Vision Models Are A Bit Too Literal

Don’t Believe Anything You Read
Several computer vision models, including OpenAI and Tesla, are proving the truth of this aphorism in some recent results. You may recall the power of a couple of pieces of electrical tape to convince a Tesla’s autopilot that a 35 MPH zone is actually an 85MPH zone, however OpenAI’s computer vision model, CLIP, is a little more gullible.
If we were to write a noun, like pizza, on a piece of paper and then stick it on a disappointingly non-pizza object, the model will happily declare that thing is indeed a pizza. These attacks have been dubbed typographical attacks, and seem to be very effective. As long as the model can identify your handwriting as being text and can match the word to a noun in it’s database, it will take your word over any existing pattern which might match the actual object.
The reason that this kind of attack can succeed is due to the flexibility of OpenAI’s computer vision model which can “recognize Spider-Man when the superhero is depicted in a photo, a sketch, or described in text“, which was the example offered to The Register. That flexibility is both the model’s greatest strength and weakness, at least for as long as text receives higher weight in these conditions; a very difficult balancing act if it is to continue to be able to associate text with images.
Try not to torment our prospective AI overlords with this too much, they may store it for later reference.
OpenAI researchers believe they have discovered a shockingly easy way to hoodwink their object-recognition software, and it requires just pen and paper to carry out.
More Tech News From Around The Web
- Furious AI Researcher Creates Site Shaming Non-Reproducible Machine Learning Papers @ Slashdot
- First Verizon, now T-Mobile: US carrier suggests folks use 2G to save battery @ The Register
- The iMac Pro has been discontinued @ Ars Technica
- AdGuard names 6,000+ web trackers that use CNAME chicanery: Feel free to feed them into your browser’s filter @ The Register
- Leo Says 57 – Anandtech’s early review of Intel Core i7-11700K @ Kitguru
- Intel CPU interconnects can be exploited by malware to leak encryption keys and other info, academic study finds @ The Register
- Google’s FLoC Is a Terrible Idea @ Slashdot
- Hot DRAM: Shortage of memory chips will continue this year, says Micron @ The Register
- The new Google Pay repeats all the same mistakes of Google Allo @ Ars Technica
- Chairs Technica: We review two new models from Anda Seat @ Ars Technica