Facebook has been awarded a patent for technology that scan through your photos, see what products you like, where you are located and more, and then send that data to advertisers.

The "Computer-vision content detection for sponsored stories" patent was filed to the U.S. patent Office by Facebook in August 27, 2015 and was published online in April 9, 2019.

Here is the brif description of the patent:

"In one embodiment, a system may receive sponsor specifications from a sponsor designating one or more concepts, monitor a plurality of activity streams for upload by users of matching multimedia objects, determine a geographic location associated with each matching multimedia object, receive a request from the sponsor associated with a specific geographic area, and send to a client device associated with the sponsor in response to the request information identifying the geographic locations associated with the respective matching multimedia objects that are within the specific geographic area."

The patent relates generally to social networking systems, and in particular, applying computer vision algorithms to user-uploaded multimedia objects to detect specific objects within the multimedia object, and promoting the uploaded multimedia object from a users' news feed to a sponsored stories area.

The patent is probably based on a new AI-powered photo scanning tool called ‘Rosetta,’ which scans texts written inside photos. The tool allows one to scan billions of photos to better learn different image texts, including brand labels.

These connections among Facebook's users are leveraged by the website to offer more relevant information to each user in view of the users' own stated interests in their connections.

Faebook typically stores a record for each users. These records may comprise information provided by the user as well as information gathered by the system related to activities or actions of the user on the system. For example, a system may require a user to enter information such as contact information, gender, preferences, interests, and the like in an initial interaction with the system, which is stored in the user's record. A user's activities on the system, such as frequency of access of particular information on the system, also provide information that can be stored in the user's record. The system may then use information provided by the user and information gathered about the user, to customize interactions of the system with the user. For example, a website selling books may keep track of a users previous purchases and provide the user with information on related books during subsequent interactions with the system. Information in a user's profile may also be used by the system to target sponsored stories that are of interest to the user.

Users interacting with the social network post stories or status updates to a live activity stream, such as a "news feed."

Sponsored or promoted stories generated from actual stories in users' newsfeeds are more likely to be viewed by users, because they generally involve interactions or suggestions by their connected friends, or fan pages that they are connected or subscribed to. However, sponsored stories require explicit, on-network interaction with a sponsor node, such as a check-in to a physical location, "liking" their hub page, commenting on their hub page, reviewing their product, and the like. A significant portion of off-network activity lacking explicit user interaction, is captured in uploaded photos.

According to the patent, if a Facebook user snaps a selfie at Starbucks and then shares that selfie on Facebook or Facebook-owned Instagram, Facebook’s technology can theoretically scan the photo and spot the Starbucks cup with the help of an “image object recognition algorithm.” Facebook can then sell that info to Starbucks, presumably alerting the coffee giant to the fact that you like its product, so it will buy ads on Facebook to sell you more McDonalds. Facebook could also boost the photo into your friends’ feeds as a sort of sponsored story.

An example included in the patent filing was Grey Goose vodka, showing that Facebook can automatically detect the vodka bottle in the photograph, figure out what brand of vodka you like, and coordinate with the brand to produce an online ad.

Facebook can also use the photos you post to track “other demographic information,” just in case Grey Goose wants to, say, figure out “the popularity of its vodka among females aged 27-35 living in urban areas,” according to the patent. Facebook can look through your photos and generate a “heat map” with adjustable filters so product owners may finely tune their query demographics.

There is no confirmation that the new Facebook patent will become a reality. Facebook wants to know what goes on in the lives of Facebook users, but the technology is already a lot invasive at this point.

Social media influence has changed the marketing landscape. Anyone with a follower count between 10,000 and 500,000 can be considered a micro-influencer, especially if he/she enjoys high audience engagement.