OpenAI Computer Vision Models Are A Bit Too Literal

Source: The Register OpenAI Computer Vision Models Are A Bit Too Literal

Don’t Believe Anything You Read

Several computer vision models, including OpenAI and Tesla, are proving the truth of this aphorism in some recent results.  You may recall the power of a couple of pieces of electrical tape to convince a Tesla’s autopilot that a 35 MPH zone is actually an 85MPH zone, however OpenAI’s computer vision model, CLIP, is a little more gullible.

If we were to write a noun, like pizza, on a piece of paper and then stick it on a disappointingly non-pizza object, the model will happily declare that thing is indeed a pizza.  These attacks have been dubbed typographical attacks, and seem to be very effective.  As long as the model can identify your handwriting as being text and can match the word to a noun in it’s database, it will take your word over any existing pattern which might match the actual object.

The reason that this kind of attack can succeed is due to the flexibility of OpenAI’s computer vision model which can “recognize Spider-Man when the superhero is depicted in a photo, a sketch, or described in text“, which was the example offered to The Register.  That flexibility is both the model’s greatest strength and weakness, at least for as long as text receives higher weight in these conditions; a very difficult balancing act if it is to continue to be able to associate text with images.

Try not to torment our prospective AI overlords with this too much, they may store it for later reference.

OpenAI researchers believe they have discovered a shockingly easy way to hoodwink their object-recognition software, and it requires just pen and paper to carry out.

Video News

About The Author

Jeremy Hellstrom

Call it K7M.com, AMDMB.com, or PC Perspective, Jeremy has been hanging out and then working with the gang here for years. Apart from the front page you might find him on the BOINC Forums or possibly the Fraggin' Frogs if he has the time.

Leave a reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Latest Podcasts

Archive & Timeline

Previous 12 months
Explore: All The Years!