Introduction

We recently worked on an integration of the “Azure Video Indexer” service, which analyzes audio and videos using AI, with “Image Relay,” a Digital Assets Management (DAM) system deployed as Software as a Service (SaaS).

The idea was for the DAM to use Video Indexer’s AI to automatically analyze and classify the videos that users upload, enabling searches from within the DAM for people, objects, conversations, etc., that appear in them.

Let’s explain how it was done.

Video Indexer

Azure AI Video Indexer is a cloud service that uses artificial intelligence (AI) to analyze and understand video content.

It extracts key information from videos, such as:

• Objects: Recognizes objects that appear and tracks them throughout the video.
• People: Identifies the people who appear in the video. It identifies famous people but can also be trained with a custom model using photos of the people we want to recognize. It can also track the detected people and the clothes they are wearing within the video.
• Scenes: Divides the video into scenes based on content changes.
• Text from images, using optical character recognition (OCR), on traffic signs and products in media files to create insights, labels, etc.

And from the audio, it can extract:

• Sentiments: Analyzes the sentiment of the video’s audio.
• Transcripts: Generates a text of what is said in the video.
• Translations: Translates the video’s audio into different languages.
• Audio Effects

Video Indexer has a portal where videos are uploaded and AI is applied to them. The portal displays each video with all the detected results:

• People
• Objects
• Labels
• Keywords in the text
• Entities
• Emotions

Each of the detected results can be located on its timeline to be viewed directly within the video.

This portal has a search engine to find videos containing certain people, labels, text in the transcript, brands, etc., from which we can create new material from the scenes we need.

To leverage its REST API, there is a developer portal where you can create API KEYS to access them.

Digital Assets Management: Image Relay

Digital Asset Management (DAM) is a type of software that centralizes the storage of all of a company’s digital assets. It acts as a “single source of truth” where teams can find any media asset created for the brand.

Benefits of DAM:

• Centralized storage and organization
• Efficient asset distribution
• Improved collaboration and workflows
• Brand consistency and control
• Time and cost savings
• Rights and permissions management

Features of a DAM platform:

• Content search and browsing
• File download and upload
• Addition of complex metadata

Image Relay is one of the most widely used SaaS DAMs on the market.

Image Relay has integrations with many third-party products: Adobe, Eventbrite, Zapier, etc., but not directly with any AI software that allows for the video classification we are looking for.

What we will use for the integration are its REST APIs.

Integration:

Metadata (tags) can be added to each Asset in Image Relay. This metadata can be used to classify them and facilitate their search.

As mentioned, we will integrate Video Indexer’s AI with Image Relay so that videos uploaded to Image Relay are automatically analyzed by Video Indexer, and the results are added as metadata to the Asset.

We use an intermediate piece of software, Azure Logic Apps, to orchestrate the integration.

Azure Logic Apps is an Azure integration service that allows you to automate business processes by creating workflows that connect applications and services in the cloud and on-premises.

When a video is uploaded or updated in Image Relay, a webhook will trigger a workflow in Logic Apps, which in turn sends the video to be indexed by Video Indexer. Asynchronously, when Video Indexer finishes analyzing that video, it will trigger another Logic Apps workflow that updates the Asset’s metadata in Image Relay with the indexing results.

As you can see in the diagram, the three pieces for the integration are:

Image Relay:

• WebHooks to send videos to be indexed by Video Indexer
• API to update Metadata when the indexing result is received

Logic Apps:

• Workflow to send the video to be indexed by Video Indexer (executed when the event occurs in Image Relay)
• Workflow to receive the indexing result and send it to Image Relay

Video Indexer:

• Indexing APIs
• Visualization of results in the Video Indexer portal

WebHook in Image Relay

For Image Relay to send an event when a video is uploaded, a WebHook must be created in the Image Relay account settings.