Transcript – How Artificial Intelligence Helps Enterprises Activate Their Media

SEASON 3 EPISODE 3 [INTRODUCTION] [0:00:01] GD: The use of AI is key in the creation of metadata. The metadata is key to enable the client to find what they are looking for. [0:00:13] MM: Welcome to Veritone’s Adventures in AI, a worldwide podcast that dives into the many ways technology and artificial intelligence is shaping our future for the better. I'm your host, Magen Mintchev. On today's episode, we will be discussing next-generation asset management and how artificial intelligence helps enterprises activate their media. Joining us is Billy Gil, Product Marketing Manager for Veritone’s Commercial SaaS Solutions. Gunnar Dedio, CEO of PROGRESS, and David Candler, Senior Director of Customer Solutions at Veritone. Veritone disclaims any responsibility for any statement of guests in the podcast. The views expressed in this podcast are those of the interviewee and do not necessarily reflect the views of Veritone or its directors, officers, or employees. [EPISODE] [0:01:02] MM: Billy Gil will kick this off by answering questions like, why do I need AI-enabled asset management? What good will it do and what can AI bring to the table? Let's have a listen. [0:01:14] BG: Why do I need AI-enabled asset management? What good does asset management do me and what can AI bring to the table? Now, media content which is unstructured is growing at a particularly high rate. Just a couple of quick facts, two and a half quintillion bytes of data is created every day. 90% of the world's data was generated in just the past two years alone, 500 hours of video content are uploaded to YouTube every minute. There are more than two and a half trillion PDFs in the world and growing. You can see that the rate of content growth isn't slowing down anytime soon. That's why organizations across industries are increasingly adopting digital asset management solutions. DAM’s streamline asset management by creating a centralized management system for digital assets. The benefits of this are myriad. Assets can easily be lost when your assets just live freely on the cloud and anyone can access them. That means, anyone can move them, or delete them, and you can reduce asset requests. Your creative department, let's say, may be fielding the same requests for logos, or videos over and over again. This way, anyone can find them easily and get back to business. It also speeds your time to market. When teams can’t find the assets they need, when they need them, they can create more and better campaigns, increase productivity and ultimately, increase revenue. When employees can't find the assets they need, that they may spend time, or even money creating and buying new assets. In fact, a recent survey by Nuxeo of sales, marketing, and creative professionals showed 75% of respondents spent time recreating something they knew existed but just couldn't find. A DAM helps you take inventory of what assets you currently have, so you can reuse and alter, rather than wasting time and money on new ones. DAMs help a remote and dispersed workforce collaborate, which pretty much speaks to everyone now over the past few years especially; gives them the ability to quickly find and share content and monetize if needed. To that last point, we’ll talk with Gunnar later about his particularly interesting use case that can apply to organizations of all kinds when it comes to doing business across locations and languages. No matter what assets your organization is producing, AI is really the key to managing it all and unlocking all of that great data, whether it's audio files from streams and broadcasts and podcasts, to sales calls, or high-res images and video, you really can't get the visibility and insights that you need without AI. AI helps you get the most out of your content by automating processes and generating insights. Some of these top areas that you probably aren't generating enough insights are audio, images, video files, and basically, anything unstructured that isn't text-based. Without AI, you can't search through visual and audio files alongside your text-based files and find everything you need. You'll also need to manually create metadata and tag it to each piece of content. AI engines are trained to automatically recognize faces and logos and objects, while transcription and translation can make your audio files easily discoverable, regardless of your language or location. On top of that, AI engines can be trained to recognize the faces and images, or other criteria that are specific to your industry and your organization. Say, your executive team, what have you, you can train your engines to recognize those people. Automated tagging speeds up the identification, segmentation, and retrieval of your digital assets, as well as helps with the compliance assessment of your assets. AI helps you discover all the great content that’s stored in your archive. Later, we'll hear about a pretty well-known organization, who was able to unearth some extraordinary moments in its archive that they didn't even know they were sitting on. With that out of the way, I'd like to introduce to you, Gunnar Dedio. A little bit about Gunnar, he grew up on the eastern side of the Berlin Wall, before founding his first film production company, LOOKSfilm in 1995. In 2019, he took over PROGRESS, one of Europe's largest archives, which contains, among other things, the entire film heritage of the German Democratic Republic, with thousands of previously unseen treasures, from life east of the Iron Curtain. PROGRESS digitizes film and photo collections from all over the world, and with the help of human and artificial intelligence, makes them accessible to the worldwide community of filmmakers through the platform, www.progress.film. If there are any historians, or film buffs out there watching, I highly recommend checking out some of the stuff they have on there. Gunnar, I just wanted to start by asking if you could walk me through how and why PROGRESS was founded and just some of that. [0:06:08] GD: The former life of PROGRESS was being a film distributor in East Germany so in the former Soviet occupation zone. This was so many times, there was only one film distributor and monopoly, the film distributor in order to distribute all new films to get out the old – the narratives out of the brains and substitute that with a new utopia of everybody's equal and live freely, etc. Then, as we know, nearly 40 years later, in 1989, the wall which surrounded this country collapsed, and this was the end of this land. It’s a monopoly of the well-being of one of the biggest European film distributors. Then PROGRESS had to look for a new sense of being and that was helping to digitalize, monetize, and bring order into the audio-visual heritage of this country, East Germany. Some years later, in 2019, I had the honor to take the management of this company and we went one step further in this process using new, more than forms of digitization and of creating metadata, of creating a platform to make this content and a lot of other contents, we in the meantime, required available to the worldwide communities of filmmakers, broadcasters, streaming platforms, museums, publishers, etc. This is the moment a bit later, where we met Veritone. Since today, we have 24,000 assets to be managed and their metadata, static metadata and dynamic metadata about the whole asset and dynamic metadata, as we call it, like timecode-based metadata on the content. Our clients coming from all over the world, and they are looking for to license archive material, to produce new films out of what we provide them. The archive is a fundamental source of raw material for the whole section of historical films, historical series. There's an enormous time and need for these contents on all public and private broadcasters and all platforms. The problem, which is there with the archive is that most of the existing archive in the 20th century is not yet digitalized. It's not enough to digitize it. You need to create metadata. The creation of the metadata is the real costly affair. This became only possible with a combination of artificial intelligence and human intelligence. This needs to be handled. This needs to be financed. The way we approach this challenge is that we go to these archives with their treasures worldwide. We propose to upfront invest in the digitization and creation of metadata. Bring this content to the worldwide market, and have a revenue share, which recruits our upfront investment into the digitization and creation of metadata. In a way, we serve to be in the middle that we enable this digitization, process in the creation of metadata process for these archives. We let them be still the owner of these assets we create. On the other hand, we serve the worldwide community of producers, podcasters, streaming platforms, museums, publishers, etc., who need desperately this content from 20th century. There is so much out there that the challenge is how to scale that in the age of platform, since in order to win this bet we do, we need to grow. We need to grow with scalable processes and with a scalable platform, not only able to handle all the incoming material, but also, able to handle the artificial intelligence to create the metadata and able to handle the process of research and sales, process and delivery process to the client. We were very happy to meet Veritone and the opportunities we found there, since it was possible for us to treat these masses of files of assets and metadata, we continue to acquire and find together with Veritone, methods of making our work scalable. Finding, for instance, procedures and workflows, which enable us to make an automatic delivery. If you want a shopping experience, which people know from their private life of buying shoes, books, or whatsoever, which is, we think their expectation, so we want them to deliver a comparable shopping experience, if you want so, in their way of a B2B business, to find what they're looking for and to be able to license it and to be able to deliver this to the clients, which is a challenge since key film abstracts are huge files. Would be too expensive to store them in a cloud and to use known ways of delivery. We had to invent quite a lot of new workflows and to combine different systems. All that was done very smoothly in this collaboration with Veritone. [0:12:35] BG: I was wondering if you could tell us a little bit about some of the benefits that you see from the solution. It sounds like, you're able to deliver your end customers marketplace, which is something that they are looking for and that they're used to using on other platforms. From an internal perspective, how has this new AI-enabled DAM solution helped you scale your business and set yourself up to continue growing your business? [0:13:04] GD: There are several layers. One layer, which seems to be very simple, but which was a complicated issue to solve was we had before a manual process of delivery. Since these are huge files, this is a really complex way to deliver a licensed piece of film to the client. This was manual work. Somebody sitting, they’re sending files from left to right. We were able to automate that so that is of course, much more cost-effective and we could shift this workforce into the creation of metadata, which is much more important for us. The second point is the use of AI is key in the creation of metadata. The metadata is key to enabling the clients to find what they are looking for. But AI develops very fast. What works this year good, we know already that next year, there will be a new generation. There's an ongoing process of using AI. For us, it was crucial to find a partner who is really AI-centered, whose basic basis of thinking is AI, since this makes the difference for us on the market. Only if a client finds what he's looking for, and we’ll license it to create this only manually with human intelligence is impossible. It's too much data. This was important. Then, it was important to us and to be able to reach out in all different language areas. It's not enough to create this metadata in one language. It needs to be translated in other languages. This is another use of the AI, and this is another way to scale. [0:15:04] BG: Well, thank you very much Gunnar. I just wanted to turn it over then to David Candler, Senior Director of Customer Solutions for Veritone, to explain a little bit more about how the digital media hub works from a more technical perspective. [0:15:22] DC: Thank you, Billy. Thank you, Gunnar. David Candler, Senior Director of Customer Solutions at Veritone. Basically, DMH, Digital Media Hub, as we call it, is an AI-powered cloud-native asset management platform. The key differentiator here is, I mean, everything we do at Veritone is really boiled down to two simple things. We create efficiencies in workflows by using the power of AI and we help content rights owners and rights holders to monetize their content. In North America, we have our own content licensing business and we represent major news and sports archives all over the world. We also provide the tools for people like Gunnar in PROGRESS to actually monetize their own content using our tools. Really, AI – I mean, Billy, you talked about some statistics earlier on. I think the one that I the best, the simplest one is 90% of all data has been created in the last two years. Really, AI is not just a nice to have. It's so fundamental going forward. There is too much data for human beings to process manually now. In the case of PROGRESS, right now we're looking at 23, 24,000 films in an archive. It's not just the films as a whole. It's the moments in the film sometimes because Gunnar and the team will also license full assets, but also moments from assets. That's the key thing is to expose these moments. If you can't see it, you can't find it, you can't do anything with it. Basically, Digital Media Hub, it's architected, hosted in AWS (Amazon Web Services), using some key web services for the workflow. Then it's plugged into aiWARE, aiWARE is Veritone’s operating system for AI. Unlike other companies, we're not just plugging into one or two engines, we've actually built an entire operating system, which has an adapter layer, an ingest layer, and a cognitive layer, which has up to 20 cognitive categories of AI. Then when you process your media through this, your media being could be live content, but it also, in this case, it's closed file content, images, videos, stills, etc., etc., audio files. Once you process that across multiple cognitive engines, whether that's facial recognition, or speech to text transcription, or logo recognition, or whatever it is, all of that data ends up in an intelligent data lake that we've built. Now the consumption amount of that data, you could be a company that already has a system and you'll use that data within your system. You’ll use our APIs to take the data into your third-party MAM, or whatever it is. We've also built a number of applications. Now, we span multiple industries, from energy, to government, to legal, media and entertainment. In this case, the application we're looking at is digital media. As I said, I've probably done this the wrong way around, but here we are, the adapter layer, the cognitive engines, the intelligent data lake and the API, or the industry applications. There you have it. You have the audio, you have video, you have images and you have text. There's the breadth of the categories we have. We have hundreds and hundreds of engines under aiWARE. Over the years, I think we've probably gone through thousands of engines. But if the reason were generally around about 300 or 400 is because really, like Gunnar said, the expansion of AI, the speed of the development means that you've got something akin to a football league. Today, there might be one engine provider out there that's top of the league. In three months to six months’ time, someone might have taken over, or there'll be some new technology. The benefit of having an operating system is you've engineered once into this ecosystem of engines, and you can very quickly switch and upgrade the engines that you're using for the benefit of your business. That's really the power of the AI. It really sets us aside. In licensing terms, for us, we have customers like CBS and PROGRESS and people like that, CNN, Bloomberg. There's so much data coming into the platforms that to enable you to monetize, you have to find it first. You have to make it as purple as possible. That's what AI really does. Just moving on to PROGRESS itself. Now, Gunnar, I'm going to bring you back in here. There's a nice picture of you with your film archive, pre-digitization there. We're really at the end of this process where content has been digitized. Now, the problem that we saw when we first started talking was PROGRESS needed to find a technology partner that would help them fulfill a wide range of short and long-term aspirations, all focused on not only monetizing but preserving very, very valuable historic archives. Using our AI-powered DAM solution, digital media hub, we're about to relaunch their global footage marketplace. We're very excited about that. We started, like all good projects, we started with a scoping exercise, where we sat down together over many days, and many hours to say, what is it that needs to happen within this solution set for you to realize your requirements and aspirations? We did some detailed documentation. Really, it was about a number of key platform developments that we identified, which would allow for a wide range of future preservation and monetization opportunities. These developments, really, they benefit progress, but they also benefit a number of our other customers as well as we go forward. The key platform developments, and Gunnar, you started talking about this, obviously, a multi-language user interface to start with. You can use Google Translate, but it's not the most accurate of technologies. Really, for progress, this was not the standard that they required. They've got buyers and sellers globally, all over the world, across Europe and the Arabic states, all over the place. What we did is we basically provided a translation table of every single UI asset in digital media. Then Gunnar’s team basically provided us with translations. We've started obviously, with German and English, and we're moving on to Arabic next, and then Latin languages to follow. When we get that translation back, we can make sure that 100% QC translation of the UI is in place for their customers. Then of course, there's the all-important metadata, whether that is asset-level metadata or timeline metadata. In that case, Gunnar and the team have done some extensive work in the past for some of their assets in some of the languages. Of course, as you expand globally, you need to fill the gaps. If you don't have Arabic metadata, what we can do is we can take the English, or the source metadata and translate that using our AI translation engines, and then fulfill. It's almost like having a table of languages that you do have in languages that you don't, and AI can help you fill those gaps by doing an automated translation. Just pausing on that bullet point, Gunnar, do you have anything to add to that first point? [0:22:37] GD: It’s exactly what you're saying. It's about the UI in to reach out to the different markets where the buyer needs to feel at home. He needs to have the same experience he would expect in private life. It is the same for the metadata. For both, it's important because we not only sell the metadata and the films, we also sell trust. People need to trust what they are buying, because they use that in films, so they need to trust that this is historically right, what is there. That's why our clients would react very sensitively on their translation because immediately, they would shy away, because they would think, if there is a bad translation, it might be bad content. That's why these precise translation, which are precise in an eye of a historian, for the UI and for the metadata is important for us. [0:23:41] DC: Excellent. Thanks, Gunnar. Just moving on to non-standard user payment methods, I should explain what non-standard is. Our standard was basically, either UK pound or US dollars, which was predominantly our business. Now as we expand internationally, we have to pick up different tax situations across Europe, different currencies, etc., etc. Basically, in the digital media platform, when you basically add the e-commerce model, it allows the users to basically search the library, put assets, or sub-clips of assets into a shopping basket, and then basically, go through a checkout series where you view a license agreement, you agree on the price, there's a set price and then some offline pricing that happens. Then you check out. It's fully integrated with things like Strike for payments and stuff like that. Then there's all the sales reporting. It's really an end-to-end full kit for monetizing your content. Of course, as we move into Germany, you've got different VAT regulations and pricing, etc. Then, of course, PROGRESS run their own pricing on their media, which we're able to populate within our database there. We did a lot of work. Then going forward, we'll do much more work on actually translating invoices and all of that stuff into different languages. I think that's really enhanced our service and certainly, has given Gunnar and the team the basic tools they need to sell. Then, of course, we need to migrate the existing users. [0:25:17] GD: David, just to remember for the non-standard, not to forget that there are clients, like the BBC will never take out a credit card to pay. They need an invoice. We need to create a workflow where we can say, this client can buy and we deliver immediately, but we trust that they will pay their invoices. They say, it's like Strike and all this – It's very modern. But when it comes to the old-fashioned invoice, modern systems, sometimes they fade out. We found a way here as well. [0:25:54] DC: Yes, absolutely. Yes, part of our open exercise is to cover the multiple ways that you actually do transact with your customers. There are more to come, as we discussed in the future, certain subscription models and stuff like that we will come against. It has really expanded our platform as well. Thanks for that. Yes, you're in business right now. You have buying customers of your content. We now have to migrate all of those users seamlessly. There can't be any downtime. This is something we've worked on. It's not only the user information, it's about the access that they've had to your existing archive, and so on. We're working on that right now. Then one coming out, which is an interesting one and Gunnar, you can probably talk about this. The multi-key frame display. You've got AI to look at moments, to find through speech-to-text, or facial recognition. You can get to a moment in a piece of content. Sometimes our customers can tell us a lot more about how their archivists search for content on the platform. This use case was to actually take every single keyframe in a video and then display it in five-second chunks, or ten-second chunks, etc., etc. It's a nice visual display of the video, so you have a visual representation of moments in the content. Gunnar, why was that so important and why is that so important in the future for your archivists? [0:27:14] GD: Because sometimes if you produce a film, or very often, if you produce the film, you are looking for a visual. It's very hard to describe what you're actually looking for. But you know it when you see it. But if you are an archive researcher, we all do have limited time. They do not have the time to watch through all the videos, even if they make it fast forward. In order to be very fast and efficient, looking through a lot of films in very short time, seeing the keyframe from every 10 seconds, for instance, gives them an overview about a whole film in a couple of minutes, because the human eye is able to grasp that very fast. Then to pick a certain frame and to say, “This is what I'm looking for.” [0:28:10] DC: Yeah, absolutely. Then, of course, Veritone digital media is cloud-based – it is hosted in the cloud. Our standard storage is Cloud Object Storage. We integrate with the likes of AWS and IBM and Microsoft, etc., etc. In this case, in certain cases, when you've got a very large volume of content and it's in 4K and beyond, sometimes the economics of the cloud still don't make sense to some people. Sometimes there's a hybrid solution. Sometimes the customer will already have a solution that they're still using. I mean, in this case, Gunnar’s team uses EditShare as their local man. Then that backs on to spinning disk and LTO, etc., etc., for their main archive. We worked with Gunnar and EditShare for a full integration here. What it means is that basically, the workflow is that the master content resides in EditShare. We get sent a proxy of an MP4, H264 proxy of the master content. That references back to the master content in EditShare. What it enables you to do on the marketplace is to view the content with all of the AI, all of the searchability, etc., plus market inpoint, and market endpoint. When you go through the sales process and it comes to the fulfillment point, then a trigger is sent back to EditShare to retrieve the master content, whether it's in 4K, or whatever, it retrieves the whole asset, or the portion of the asset up into the cloud. In this case, it's Amazon S3, where the customer can then fulfill or accelerated using accelerated download can actually fulfill the content. Then the master content is deleted after a while from the work in progress S3 bucket. We've built this integration to basically, allow PROGRESS to keep their 4K masters on-premise, in their platform, but actually view it, preview it, collaborate with it and then ultimately, buy it and then fulfill that master assets through one single user interface, which is digital media. I mean, I guess, Gunnar, this is a bit of a game changer for you in terms of these in your workplace. [0:30:19] GD: Yeah, absolutely. Because this was done manually until now. In a digital age, this does not make sense to fulfill and early, the delivery of digital assets. It is costly since you need a specialist to do that. This is indeed a game changer because on this way, we can put our resources into things that really matter, which for us, the digitization, and especially the creation of metadata. [0:30:49] DC: Yeah. Then finally, digital media have, like all good platforms have a robust set of APIs. What that allows you to do? I mean, digital media has a configurable UI. You can brand it with your own backgrounds and your own logos and your fonts and accent colors and all of that stuff. Some customers would like to have a secondary or like to design their own UI. That's when they use the API. Some good use cases for us are the likes of Bloomberg Mercury, and charlierose.com, where they've got design agencies who have built a user interface but use our APIs to use all of our key functionality, like our search, our video player, upload, download our metadata index, search, index, etc., etc. I think going forward in the future, we'll have a marketplace that will open up that's using the Digital Media Hub user interface. Gunnar, you've got some exciting ideas for the future where you might be taking the API to access the same content for different business ventures. [0:31:51] GD: Yeah. Because basically, what we realized is that we do a B2B offer, where professional clients are there to license from the archives. We offer them. We realized there are thousands and thousands of people knocking on our door, which are private people. They're just interested. They’re students, professors, people making family research. A lot of different reasons brings them to us, they find us and they want us to make research in our huge archive. On the beginning, we said, “Thank you, but really, we don't have the egress for you. We don't have the time. This is not made for you.” Since it was more and more, we said, “Okay, we need to create a different way of access to this huge content we have, because these people, they are willing and able to pay for this access. Why don't use this other way of monetization to create another stream of income, which we can reinvest in digitization and the creation of metadata of a new archive?” This is the new axis. We are now building together with Veritone, a new surface, a completely different way of looking at the same content, a different way of monetization, where we let’s say, just react on what the market asks us. [0:33:27] DC: That's brilliant. Okay. Well, thank you very much, Gunnar. We're so excited to work with you and launch your new platform in the near future. I've just got a couple of other use cases that I just want to talk through. A couple of our customers is another European customer, Inter Milan. Basically, this is a different use case, where Inter Milan, well, again, had their own asset management platform. In this case, they wanted to use our AI. We talked earlier on about the aiWARE operating system for artificial intelligence. What we've done here is we've plugged AI into their third-party system, which in this case is a media asset management platform called Evolving Systems. What actually happens is as media is ingested into the Evolve in Zoom MAM, basically, a proxy of the asset is created. That is sent to Veritone aiWARE, automatically in the workflow with a manifest. That manifest is information containing which cognitive engines that we want it to process a media against. In the case of Inter Milan, we are looking at facial recognition and we're looking at transcription. I listen to the commentary. In this case, some of it, I think, most of it is in Italian, and then logo recognition, or brand recognition, because obviously, sport is all about rights and sponsorship, of course. Finding those moments is important. Basically, again, with sports, fan demand is huge. Inter Milan produces massive amounts of video and audio content. They need to not only manage this but have the ability to easily find it, distribute, and monetize it. They've got their MAM. The aiWARE cognitive services really gives them that discoverability, so they can have that quick find and quick turnaround because sport is all about the moment. Basically, what we've got really is we've processed over 300 terabytes of content with over 1,000 hours of video as a result of the partnership. We've made near on 65 to 70 million detections using our AI there. Really, that's a very slick use case of where AI is not just built for Veritone applications, AI can stand alone. Where we talked about that consumption level at the top of the technical stack, this is using our robust API to give third parties the ability to access and use our AI on their own systems. That's Inter Milan. Then another great use case, I mean, check out the website, the customer made a really great video use case study and they can talk in a lot more detail about what they did, but the San Francisco Giants, they have a 60-year-old archive, and it is aging. Gunnar, you’ll know this, obviously, when some physical tapes and film stock, it does age and it eventually disappears, decays and disappears. It's really important to move on digitization to really preserve these archives. As you look at archive projects, video without any metadata around, or any reference is very hard to surface, even if you haven't digitized it. Basically, we've used AI to allow the Giants to basically, automatically tag key moments in the sport. This would have been a manual approach and this would have been years and years and years. That really reminds me of one of the fundamental messages that we make and it's actually the Veritone loop theory. If you have to do manual processes, listen, go find me all of the Giant’s logos in this archive. You're talking about decades of work. That would be having a person in the loop, a very manual process. Going completely out of the loop is where AI has taken over the human process. For many customers, that's not going to be potentially in our lifetime. What we're focused on is actually empowering our customers sit on the loop. If I asked the same question, “Go find me all of the San Francisco logos in the archive” AI could do that in a matter of days by using logo recognition. What that's doing is it's empowering the operator not to get into the loop, again, on the loop and take that really important data and do something with it. I think in the progress use case, it's putting their customers on the loop, they've now got the power to use that AI to discover the content and use it in a downstream use case. In this case, we used aiWARE and digital media hub for the San Francisco Giants. It enables us to run multiple cognitive entities across their content as it is ingested into digital media. It opens up new monetization opportunities for the Giants. Again, this is accomplished in days, instead of months. It's done, I do hate to say, but yes, I didn't have to hire 15 people. Those 15 people didn't exist, but the math actually works out, they would have had to have hired 15 additional staff members just to log content, to make use of it in the manner that they would like. Another happy customer there. [END OF EPISODE] [0:38:31] MM: We hope you enjoyed today's episode talking all about asset management and how AI helps organizations activate their media. Don't forget to subscribe, rate and review and share this podcast. This has been another episode of Veritone’s Adventures in AI, a worldwide podcast that dives into the many ways technology and artificial intelligence is shaping our future for the better. Talk to you next time. [END] Subscribe