Analyze images with Content Understanding | AI-103 | Episode 24

Azure Content Understanding converts unstructured multimodal content into structured, schema-defined data. For images, think beyond “describe this picture”: an analyzer can extract exactly the fields your application needs and return them as structured output.

Core mental model

Content → Analyzer → Schema → Structured output

An analyzer is the reusable processing configuration. It defines the modality (image, document, audio, video), what information to extract, how the output is structured, and potentially which models perform the processing. Use prebuilt analyzers for common scenarios or build custom analyzers with your own fields.

For example:

Image
  ↓
Custom image analyzer
  ↓
Schema:
  description : string
  tags        : array[string]
  ↓
JSON:
{
  "description": "A giraffe in a savannah",
  "tags": ["giraffe", "grass", "savannah"]
}

Studio vs. Foundry — know the relationship

They are not separate AI services:

Microsoft Foundry resource → Content Understanding service → Studio / Foundry experiences

• Microsoft Foundry provides the broader AI development environment and a Content Understanding playground for running supported analyzers.

• Content Understanding Studio is the specialized experience for Content Understanding and supports the full prebuilt analyzer catalog, creating custom analyzers, schema authoring, and in-context learning/data labeling.

• Both use the same underlying service, Foundry resource, and Azure authentication.

Remember: Foundry = broader AI platform; Studio = deeper Content Understanding authoring experience.

From analyzer to application

After configuring an analyzer, call it through the Content Understanding API/SDK:

client = ContentUnderstandingClient(endpoint, credential)

poller = client.begin_analyze_binary(
    analyzer_id="image-analyzer",
    binary_input=image_bytes
)

result = poller.result()
# → structured fields defined by the analyzer

Authentication can use Microsoft Entra ID or an API key. Content Understanding requires a Microsoft Foundry resource, with the required model deployments configured for analyzers that use generative models.

Key distinction: use Content Understanding when you need repeatable extraction of structured information from multimodal content, rather than only an unconstrained natural-language response.