How are pictures and videos shaping Media Intelligence?
Pictures and videos are changing our whole living environment. Visual elements are affecting the way we communicate and even the way we see the world. Media Intelligence and analysis is keeping up with the progress. Visual analysis is still mostly human made but machine learning is quickly closing up on human performance.
Pictures and videos are not just a part of media but also changing media as we know it. Visual elements are important elements not only in social media but also in editorial media. News channels have been using online videos for a while now and they are using these elements increasingly.
Internet media company BuzzFeed serves as a great example of today’s media and where we are headed. BuzzFeed was just valued at more than three times as much as the Washington Post. The barriers between news and entertainment are breaking down and visual elements are playing a part in all of this.
This is also changing the way companies communicate. Content marketing has taken its place in many marketing plans made today. According to a report made by the Content Marketing Institute, 93% of B2B marketers use content marketing as part of their marketing plan. Amongst the content shared and used for marketing, videos and pictures are more and more vital. But how does this affection for everything visual show when it comes to media intelligence and publicity analysis?
First of all, visual elements are not entirely new. Newspapers and magazines have used pictures for almost as long as they’ve existed and television has already broadcasted for decades. But what is new is the constant and overwhelming flow of videos and pictures online in both editorial and social media. How to measure these arenas? This new visual world presents its own kind of challenge.
Is the gap between machine learning and human performance closing?
A lot of the visual measurement and analysis has been done by humans and a lot still is. Even as I write, M-Brain’s Content Specialists are monitoring a vast variety of news broadcasts both on TV and radio picking the news pieces that matter to our clients.
Our Media Analysts look at the visual and text elements as a whole while scanning through newspapers, magazines and other sources. And when it comes to analyzing the content, pictures and broadcasts are valued higher. As such, visual elements are already an important part of business intelligence and publicity analysis.
But Machine Learning techniques are making an impact in the business applications of image analysis. This development presents opportunities on the more technical side of media monitoring. Just this March Facebook’s AI research group reported a major improvement in their face-processing software. After this improvement the software has been able to do almost as accurate recognition as a human could.
When asked whether two unfamiliar photos of people’s faces show the same person, a human would be right 97.53 percent of the time. The new improved software called DeepFace can score 97.25 percent on the same challenge. This kind of artificial intelligence is hardly in everyday use yet. Even Facebook says that DeepFace remains purely a research project for the time being. But this gives an idea where we might be headed.
According to Mr Kimmo Valtonen, the Acting CEO of M-Brain, in the world of media measurement these kind of state-of-the-art techniques are not widely used. There are some methods, such as the face recognition described above, which are giving great results. But these methods are mostly concentrated on one specific area, such as faces, and are results of a long-lasting research.
“Our customers do show growing interest in pictures and videos. At the moment we mostly monitor pictures and videos by automated analysis of the text around it. Actual image analysis is currently still human made here at M-Brain. When it comes to its automation, I doubt the quality of human made analysis can be seriously challenged in the foreseeable future ““ but quite soon there will be an increasing amount of tools to make human analysis more scalable”, says Kimmo Valtonen.
What is the future of visual analysis?
There are quite a few obstacles to overcome before automated picture analysis can become everyday practice. For example it takes a lot of storage capacity to store pictures and videos, a lot more than storing text elements. This is why clientele must be ready to pay more for these services for them to develop. So far clientele has not been very keen on investing in this kind of analysis.
“We have already tried, for example, logo spotting with state-of-the-art methods. This is a niche problem in the sense that it involves a static and quite simple object. Related problems are for example the different view angles and shades on the picture. It turned out to be feasible to recognize acceptably given examples of logos in an experimental setup, but discovering the best way to bring such methods into widespread use in our commercial offering is still work in progress”, Mr Valtonen says.
When asked what Mr Valtonen sees in the future of visual analysis he refers to Big Data and how it’s not only about the amount of data you have but about how you combine this data.
“I believe the research will head more and more towards studying visual and text elements together as a whole. These elements should be analyzed as inseparably as possible.”
“Thinking of the DeepFace mentioned earlier and similar applications, comments and captions naturally convey information about the object in the picture, for example a person. It is already possible to roughly analyze a person’s emotions from an image. With this kind of combined teaching material from text and picture, automation would then try to learn what textual, emotional and visual elements appear together and in what manner. In pure text analysis this is already possible at a certain level. To give a concrete example, such a method would be able to tell what emotions people seem to feel when using a given B2C product. Putting all this together in a commercial offering is still ongoing applied research work”, Kimmo Valtonen describes the possibilities of machine learning and automated analysis.
Staying on the cutting edge
The media landscape is constantly changing. Agents working in the media field must keep up with these changes and understand how these changes affect our whole living environment. According to M-Brain’s Acting CEO Mr Kimmo Valtonen, an example of these changes is how our language is evolving through visual elements.
“What’s also really interesting is how pictures affect language and interplay with it. For example on Instagram the picture is the main content and this means that the language is stripped from all unnecessary elements and takes shape as tags which do not necessarily carry a lot of independent meaning without the picture.”
According to Mr Valtonen, M-Brain wants to stay on the cutting edge when it comes to image analysis methodology.
“M-Brain, having still a relatively compact R&D team, we cannot compete with major players in the amount of development we do, so from the start we have opted for membership in research projects of the highest caliber. At the moment we are actively involved in three such projects, of which Data2Intelligence, a SHOK program, involves partners doing state-of-the-art image analysis. M-Brain contributes text analysis research and at the same time benefits from access to the latest research into fields such as image and video analysis, where it is not cost-efficient for us to focus in the pre-application stage.”
By Sanna Tolmunen
348/365 – the backend of nowhere
The iPad of Manhattanhenge
Marketing & Communications