Multimodal AI
AI using several information types, such as text, images or sound, in inputs or outputs.
What is multimodal AI?
Multimodal AI processes or generates more than one type of information, such as text, images or sound. A model can consider different inputs together yet output only text. Supported combinations depend on the specific model and its integration. The term does not promise understanding of every input type or generation of every output type.
In a fictional service case, a model receives an equipment photo and an accompanying written note. Both sources are intended to help assessment together. If the nameplate is unreadable, the workflow needs clarification rather than an unsupported equipment identifier. The team checks whether actual image and text content supports the results. Extra input types may add useful context but do not replace that check.
Sources
Related terms
- Neural networkA model of connected computational nodes with adjustable parameters.
- Large language modelA large language model processes language using learned patterns. Fluent text proves neither factual accuracy nor access to current business data.
- Speech-to-textConverting spoken language in audio data into written text.
- Text-to-speechProducing synthetic spoken output from written text.
- Optical character recognitionOptical character recognition makes writing in images machine-processable. Reading an amount is not the same as checking an invoice against business rules.
- Model cardA model card documents an AI model’s purpose, foundations, evaluation conditions and known limitations for a particular version.
Articles on this topic
AI & Automation with ASES
AI assistants and automated workflows