Course outline

18.1

Vision, Speech and Language Services

Rekognition, Transcribe, Polly, Translate, Lex and Connect, Comprehend and Comprehend Medical.

26 min Reading + animations

Tip: tap or hover any dotted-underlined term to see what it means.

By the end of this lesson you will be able to

  • Explain what a pre-trained AI service is, the difference between training and inference, and why the exam prefers a managed AI service over building your own model.
  • Use Amazon Rekognition for labels, text, faces, face search and content moderation, and add human review with Amazon Augmented AI (A2I) when the model is unsure.
  • Turn speech into text with Amazon Transcribe (speaker labels, custom vocabulary, PII redaction, language identification) and text into speech with Amazon Polly (voice engines, lexicons, SSML).
  • Translate content with Amazon Translate, and build a conversational IVR or chatbot with Amazon Lex and Amazon Connect.
  • Analyse text with Amazon Comprehend (language, entities, key phrases, sentiment including Mixed, PII) and clinical text with Amazon Comprehend Medical (medications, conditions, PHI, medical codes).
  • Pick the right service from a scenario’s keywords, and tell apart the look-alikes: Transcribe vs Polly, Comprehend vs Comprehend Medical, Rekognition text vs Textract.

#AI services: intelligence you call, not build

#Training, inference and “pre-trained”

A machine learning (ML) model is a program whose behaviour was learned from examples instead of written by hand. Building one has two phases:

  • Training: show the model millions of labelled examples (photos marked “cat” or “car”, hours of audio with their transcripts) until it learns the patterns. This needs data scientists, a lot of data, GPUs and weeks of work.
  • Inference: use the trained model on new input: give it a new photo and get back “car, 97%”. This is fast and cheap per request.

A pre-trained AI service is a model that AWS has already trained on huge datasets and runs for you behind an API. You only do inference: send a photo, an audio file or a paragraph, get a structured answer back. No data to label, no model to train, no servers or GPUs to manage, and the model keeps improving without any work from you.

This lesson covers the services that see (Rekognition), listen (Transcribe), speak (Polly), translate (Translate), talk with customers (Lex and Connect) and read (Comprehend and Comprehend Medical). Lesson 18.2 covers building your own models with SageMaker AI, generative AI with Bedrock, and the services for search, recommendations and documents (Kendra, Personalize, Textract), plus a cheat sheet of the whole chapter.

#What you still do, and what you pay

“No machine learning knowledge needed” is true; “no coding needed” is not. You call these services from your application through the AWS SDK, the CLI or the API (and some have console demos), you decide what to do with the answer, you set confidence thresholds, and you handle errors and costs. What you do not need is ML expertise: no algorithms, no training data, no tuning.

They are serverless and pay per use, each with its own unit:

ServiceYou pay perExample price (US East, late 2026)
Rekognition (images)image analysedabout $0.001 per image for the first million a month
Rekognition (stored video)minute of videoabout $0.10 per minute for label detection
Transcribesecond of audio (no minimum)a few cents or less per minute, depending on volume tier
Pollycharacter spoken$4 (standard), $16 (neural), $30 (generative), $100 (long-form) per million characters
Translatecharacter translatedabout $15 per million characters
Comprehendunit of 100 characters (minimum 3 units per request)about $0.0001 per unit for the standard APIs
Lextext or speech requestfractions of a cent per request
Connectminute of usage (plus phone number and telephony)per minute; no seats or upfront payment

Prices change and differ by Region, so check the pricing pages; the exam only expects you to know the model: no upfront cost, pay for what you send.

#The services in this lesson

InputTaskService
Image or videoObjects, scenes, text, faces, unsafe content, celebritiesRekognition
Audio (speech)Speech → text, subtitles, call transcriptsTranscribe
TextText → lifelike speechPolly
TextLanguage A → language BTranslate
A customer talking or typingUnderstand intent, answer, actLex (+ Connect for the phone contact centre)
TextLanguage, entities, key phrases, sentiment, PIIComprehend
Clinical textMedications, dosages, conditions, PHI, medical codesComprehend Medical

#Amazon Rekognition: images and video

#What Rekognition finds

Amazon Rekognition is AWS’s pre-trained computer vision service: it finds objects, people, text and scenes in images and videos. Each feature is an API call:

  • Labels (DetectLabels): objects, scenes, activities and landmarks (“Flood”, “Car”, “Wedding”, “Beach”), each with a confidence percentage and, for many objects, a bounding box. Used to tag and search a photo library.
  • Text detection (DetectText): short text in ordinary images: shop signs, number plates, captions on a video frame.
  • Face detection and analysis (DetectFaces): where faces are, plus attributes such as an estimated age range, smile, eyes open, glasses, beard and emotions.
  • Face search and verification: compare two faces (CompareFaces, 1:1 verification) or search a face against a stored collection of known faces (1:N search).
  • Celebrity recognition (RecognizeCelebrities): well-known people from public life, sports and entertainment.
  • Content moderation (DetectModerationLabels): inappropriate, unwanted or offensive content (explicit nudity, violence, drugs, hate symbols…).
  • PPE detection: whether people wear face, hand and head covers, for safety checks on a construction site or factory floor.
  • Custom Labels: train Rekognition on a small set (dozens to hundreds) of your own labelled images to recognise things the pre-trained model does not know (your products, a defect on your production line), still without ML expertise.

Typical use cases: labelling a media library, moderating user uploads on social media, e-commerce, broadcast and advertising, detecting text in images, face-based user verification, counting people, and celebrity recognition for media archives.

Face analysis returns estimates from appearance. An age range of 29–37 or an emotion such as “calm” is the model’s guess about how a face looks, not a fact about the person: Rekognition does not know anyone’s real age or how they feel. AWS’s own guidance is not to use these attributes alone for decisions about people.

Face search works in two steps:

  1. Index: create a collection and add known faces with IndexFaces. Rekognition stores a numeric face vector (a mathematical description of the face), not the image.
  2. Search: for a new image, SearchFacesByImage compares the largest face with the collection and returns matches above a face match threshold with a similarity score. CompareFaces compares two images directly, for example a selfie against an ID card photo during sign-up.

The threshold is a trade-off. A low threshold finds more matches but more of them are wrong; a high one (95 or 99 for anything that affects people) misses some but rarely confuses two people. Rekognition returns similarity, not identity: decisions such as letting someone into a building or a police lookup need a high threshold and a human check.

#Content moderation, thresholds and human review

Moderation is the most common Rekognition question. DetectModerationLabels returns labels in a hierarchy (for example “Violence” → “Graphic Violence”), each with a confidence. You set a minimum confidence: labels below it are not returned at all (the default is 50%). Social media, broadcast, advertising and e-commerce use it to create a safer experience and to help meet regulations.

A good design uses two thresholds:

  • Above a high line (say 90%): block automatically.
  • Below the minimum confidence: publish automatically.
  • In between, the grey zone: send to a person. Amazon Augmented AI (A2I) adds a human review loop to ML predictions. It is built into Rekognition’s moderation API: you define a human loop with activation conditions (“any label between 60 and 90”), and A2I sends those images to a work team: your own staff (a private workforce) or a vendor. The answers are written to S3.

Only the unsure few percent cost human time; the rest stays automatic. A2I also works with Textract and, with a custom template, with any model (lesson 18.2).

#Video and custom labels

For stored video in S3, Rekognition Video works asynchronously: StartLabelDetection, StartContentModeration, StartFaceSearch and others start a job, Rekognition publishes the completion status to an SNS topic, and you fetch the results with the matching Get… call. Never wait for a long video inside a Lambda function: Lambda stops after 15 minutes (lesson 15.1). Start the job, return, and let the SNS notification trigger the next step.

If the pre-trained labels do not cover what you need (a specific defect on fabric, your own product packaging), try Rekognition Custom Labels before building a model in SageMaker AI: you upload and label as few as a few dozen images and Rekognition trains the model for you.

Rekognition at a Dhaka news portal

Reporters and readers upload photos all day. A Lambda function sends each new photo to Rekognition, which tags it, checks it for unsafe content and recognises the portal’s own staff. Watch the normal path, the grey zone that goes to human reviewers, and a face that does not match.

Scenario

One API call returns what is in the photo, with a confidence score for each label. No model to train.

Upload app
Photo bucketnew objects
Lambdacalls the API
Rekognitionpre-trained
Photo tagsDynamoDB
Face collectionstaff faces
A2I reviewhuman reviewers
PUT photoevent
Step 1 of 5
1

A reporter uploads a photo of a flooded street in Mirpur. The object lands in the S3 bucket, and an S3 event notification invokes the Lambda function (lesson 10.7).

#Amazon Transcribe: speech to text

#What Transcribe does

Amazon Transcribe converts speech to text with automatic speech recognition (ASR), a deep-learning process. Send it an audio or video file (or a live stream) and it returns a transcript with every word, its start and end time and a confidence score. Use cases:

  • Transcribe customer service calls for search, quality checks and analytics.
  • Closed captions and subtitles for videos, built from the word timestamps.
  • Metadata for media archives: make thousands of hours of interviews searchable by what was said.

#Accuracy and privacy features

  • Speaker diarization (speaker labels): separates who spoke when (spk_0, spk_1…, up to 30 speakers). For two-party calls recorded on two channels, channel identification separates agent and customer by channel instead.
  • Custom vocabulary: a list of words and phrases (brand names, product names, technical terms) with how they sound and how to write them. Fixes “bee cash” → “bKash” without any training.
  • Custom language models: for a whole domain with its own way of speaking (banking, law, a regional accent), train an extension of the model on your own text. Use a vocabulary for a handful of names, a custom language model for a domain.
  • PII redaction: identifies personally identifiable information (card numbers, bank account numbers, phone numbers, names, addresses…) and replaces it with [PII] in the transcript. You can get only the redacted transcript, or both redacted and unredacted. Redaction is supported only in some languages (mainly English dialects, plus a few others), and AWS warns it may miss some instances, so review it for regulated data.
  • Automatic language identification: when you do not know the language, Transcribe detects it (optionally from a list of candidates); multi-language identification handles audio that switches language mid-way.
  • Vocabulary filtering: mask or remove words you never want in a transcript (profanity, for example).

#Batch, streaming and the specialised versions

  • Batch (StartTranscriptionJob): for files in S3. It is asynchronous: start the job and return; the result is written to S3 and a job state change event goes to EventBridge (chapter 19), which can trigger the next step.
  • Streaming: for live audio (live captions, agent assistance during a call). The application sends audio over HTTP/2 or WebSocket and receives text within about a second. No file, no job.
  • Transcribe Call Analytics: built for contact centres: transcripts plus sentiment per speaker, talk time, interruptions, issue detection and call summaries.
  • Transcribe Medical: trained on medical vocabulary for doctors’ dictation and patient conversations, HIPAA eligible; its text often goes to Comprehend Medical (below).

Transcribing a wallet’s call recordings

A bKash-style mobile wallet records every call to its English support line. Each new recording in S3 starts a Transcribe job. Follow a job end to end, then the three things that go wrong in real projects.

Scenario

A batch job is asynchronous: start it, get told when it is done, read the result from S3.

RecordingsS3
Lambdastarts jobs
Transcribespeech to text
EventBridgejob events
TranscriptsJSON
object created
Step 1 of 5
1

A 6-minute recording of a customer and an agent lands in the bucket, and the event notification invokes the function.

#Amazon Polly: text to speech

#Voices and engines

Amazon Polly turns text into lifelike speech with deep learning, so you can build applications that talk: read-aloud buttons on articles, blogs and e-books, voice prompts in phone systems, announcements, e-learning narration, accessibility for people who cannot read a screen. You choose a voice (dozens of voices in more than 40 languages and language variants) and an engine:

EngineWhat it isPrice per million characters
StandardThe original concatenative voices; supports every SSML tag$4
NeuralNeural TTS: far more natural; most new voices$16
Long-formExpressive voices built for long content such as articles and audiobooks$100
GenerativePolly’s largest model (2024): the most human-like, conversational voices, expanded through 2025–26$30

Polly has no Bengali voice today; for Bangladesh, English (including Indian English voices) and Hindi are available. You pay per character; the same text gives the same audio, so generate it once and cache it in S3 instead of calling Polly for every listener.

#Lexicons or SSML?

Two tools control how Polly says things, and the exam likes to contrast them:

  • Pronunciation lexicons customise how specific words are pronounced everywhere. You upload a lexicon (an XML file in the standard pronunciation lexicon format) once with PutLexicon and name it in each SynthesizeSpeech call (up to five per call). Use them for acronyms (AWS → “Amazon Web Services”, BUET → “Bu-et”) and stylised words (a brand like “Bazar365” → “Bazar three-sixty-five”, or the slides’ example of a name spelled with numbers in place of letters).
  • SSML (Speech Synthesis Markup Language) marks up one particular text for more control: <break> for pauses, <emphasis> to emphasise words, <phoneme> for an exact phonetic pronunciation, <prosody> for rate, pitch and volume, <say-as> for dates and numbers, <amazon:auto-breaths> for breathing sounds, <amazon:effect name="whispered"> for whispering, and <amazon:domain name="news"> for the newscaster speaking style.

#Long texts and speech marks

  • SynthesizeSpeech returns the audio directly and accepts up to 3,000 billed characters (6,000 in total, with SSML tags) per request.
  • StartSpeechSynthesisTask is asynchronous, accepts up to 100,000 billed characters, writes the result to S3 and can notify an SNS topic when done. Use it for long articles and audiobooks.
  • Speech marks are metadata returned alongside (or instead of) audio: when each sentence and word starts, and visemes (mouth shapes). They are used to highlight words as they are spoken and to lip-sync animated characters.

A newspaper that reads itself aloud

An English-language daily in Dhaka adds a “Listen” button to every article. The app sends the text to Polly and plays the audio. Watch how a lexicon fixes a word Polly says wrong, what SSML adds, and why long articles need a different call.

Scenario

Text in, audio out, in one call. You pick a voice and an engine.

News appbackend
Pollytext to speech
Audio bucketaudio files
Listener
synthesize-speech
Step 1 of 3
1

The app sends a 900-character news summary with voice Kajal (Indian English) and the neural engine. Neural voices sound far more natural than the older standard engine.

#Amazon Translate

#How Translate works

Amazon Translate gives natural and accurate language translation using neural machine translation (NMT): a neural network reads the whole sentence and its context, so the result reads like a person wrote it, not like word-by-word substitution. It supports 75 languages (Bengali included) and can detect the source language automatically.

  • Real-time (TranslateText, and TranslateDocument for a single file): a few thousand bytes per call, answered in well under a second. For chat messages, product listings, reviews, support tickets.
  • Batch (StartTextTranslationJob): translates large volumes of documents stored in S3 (plain text, HTML and Office documents) into one or more languages asynchronously, writing the results to S3 and keeping the formatting.

The job of Translate is to localise content (websites, applications, help centres) for international users and to translate large volumes of user-generated content automatically: comments, reviews, chats between buyers and sellers who do not share a language.

#Custom terminology and localisation

  • Custom terminology: a CSV or TSV file of source terms and how they must come out. Brand names stay as they are (Apex stays “Apex”, not the Bengali word for “peak”), product names and domain terms are translated consistently. You name the terminology in each request.
  • Active Custom Translation: for a whole domain’s style (legal, medical, your company’s tone), you provide parallel data (pairs of sentences in both languages) and Translate adapts its output to them, still without you training a model.
  • Settings: formality (formal or informal tone, for some languages) and profanity masking.

Translating a marketplace

A Daraz-style marketplace lets sellers write listings in Bengali or English and shows each buyer the language they chose. Watch a real-time translation, a brand name that gets translated when it should not, and a batch job for manuals.

Scenario

Short text in, translation out, in one synchronous call.

Seller
Shop backend
TranslateNMT
Docs bucket
listing in Bengali
Step 1 of 3
1

A seller in Rajshahi writes a listing in Bengali: “খাঁটি রাজশাহীর আম, ৫ কেজির বাক্স”.

#Amazon Lex and Amazon Connect

#Lex: the chatbot builder

Amazon Lex builds conversational interfaces: chatbots and voice bots. It runs on the same deep-learning technology that powers Amazon Alexa and combines:

  • ASR (automatic speech recognition): speech → text, for voice bots.
  • NLU (natural language understanding): recognises the intent behind the words (“buy a data pack”, “book an appointment”) however it is phrased, and fills slots (the details: pack size, date, branch), asking follow-up questions for any slot that is missing.

When the intent is complete, Lex invokes a Lambda function (fulfilment) that does the real work: query a database, call a CRM, book the appointment. Lex is not only for call centres: the same bot answers on a website chat window, in a mobile app, on Slack, Facebook Messenger or SMS. If no intent matches well enough, Lex returns the built-in fallback intent, so your flow can apologise or transfer to a human instead of guessing. The current version is Lex V2 (Lex V1 reached end of support on 15 September 2025), and recent versions add generative AI features through Amazon Bedrock (lesson 18.2).

#Connect: the cloud contact centre

Amazon Connect is a cloud-based virtual contact centre: it gives you phone numbers, receives (and makes) calls, and routes them through contact flows (drag-and-drop scripts of what happens on a call) to queues and human agents, who work from a browser. It is omnichannel: voice, chat, tasks and email in one place. It integrates with CRM systems (Salesforce, Zendesk or your own) and other AWS services, and has no upfront payments: you pay per minute of use, which AWS markets as up to 80% cheaper than traditional contact-centre systems.

#How they work together

An IVR (interactive voice response) is the automated part of a phone line. The old kind says “press 1 for balance”. With Connect and Lex, callers just say what they want:

  1. A phone call reaches Connect, which runs the contact flow.
  2. The flow streams the caller’s speech to a Lex bot, which recognises the intent (“schedule an appointment”).
  3. Lex invokes a Lambda function, which schedules it in the CRM.
  4. The answer goes back through Lex and is spoken to the caller by Connect (with a Polly voice).
  5. If Lex does not understand, the fallback intent makes the flow transfer the call to a human agent, who sees what the caller already said.

A telecom’s IVR on Connect and Lex

A mobile operator replaces its “press 1, press 2” phone menu. Callers just say what they want. Connect runs the contact centre, Lex understands the words, Lambda does the work. Watch a recognised request, one Lex cannot understand, and the same bot on the website chat.

Scenario

Phone call → Connect → Lex intent → Lambda → CRM, and the answer is spoken back.

Callerdials 121
Connectcontact flow
LexASR + NLU
Lambdafulfilment
Human agent
Web chat
CRMcustomer data
call
Step 1 of 5
1

A customer calls 121. Connect answers and runs the contact flow: the drag-and-drop script of the call. Its first block hands the caller to a Lex bot.

#Amazon Comprehend: understanding text

#What Comprehend finds

Amazon Comprehend is a fully managed, serverless natural language processing (NLP) service: it uses machine learning to find insights and relationships in text. For any text (a review, an email, a support ticket, a news article) it can return:

  • Dominant language (DetectDominantLanguage): which language the text is in, from about 100 languages.
  • Entities (DetectEntities): people, places (locations), organisations, brands and commercial items, dates, quantities, events, titles.
  • Key phrases (DetectKeyPhrases): the noun phrases that carry the meaning (“the battery”, “the seller”).
  • Sentiment (DetectSentiment): Positive, Negative, Neutral or Mixed, with a score for each. Targeted sentiment gives the sentiment towards each entity (“camera: positive, battery: negative”).
  • Syntax (DetectSyntax): tokenization (splitting into words) and parts of speech (noun, verb, adjective…).
  • PII (DetectPiiEntities): names, addresses, phone numbers, bank account and card numbers, national ID numbers and more, with their positions, so you can mask them; batch jobs can write a redacted copy.

Each API works in real time on one document (a few KB) or a small batch (up to 25 documents), and as asynchronous jobs on large collections of documents in S3.

#Custom models, languages and limits

  • Custom classification: train a classifier on your own labelled examples (“refund request”, “delivery complaint”, “praise”) to route tickets, without ML expertise.
  • Custom entity recognition: teach Comprehend entity types it does not know (your product codes, policy numbers).
  • Topic modelling: groups a collection of documents by the topics it discovers (the slides’ “organise articles by topic”). See the side note below for its status.
  • Languages: sentiment, entities and key phrases support about a dozen languages (English, Spanish, French, German, Italian, Portuguese, Arabic, Hindi, Japanese, Korean, Chinese); PII detection only English and Spanish; Bengali is detected but not analysed. For a Bengali review: Translate to English first, then Comprehend.

Comprehend lab

Pick a text, then switch on Comprehend’s analyses one at a time. Each button is one API call on the same text.

1. Text

2. Analyses

I ordered a Walton fridge from Daraz on 3 March and it reached Sylhet in two days. Great service!

Switch on an analysis to see its result.

#Amazon Comprehend Medical

#Clinical text and PHI

Amazon Comprehend Medical is the version of Comprehend trained on unstructured clinical text: physicians’ notes, discharge summaries, test results, case notes, prescriptions. It returns:

  • Medical entities (DetectEntitiesV2): medications, medical conditions, tests, treatments and procedures, anatomy, with attributes linked to them: dosage, frequency, route, strength (“metformin” → 500 mg, twice daily, oral). The link between a dose and its drug is what plain Comprehend cannot give.
  • PHI (DetectPHI): protected health information (name, age, address, phone, ID and record numbers, dates) so you can remove it before data leaves the clinic.
  • Ontology linking: maps entities to standard medical codes: ICD-10-CM for conditions (InferICD10CM, type 2 diabetes → E11.9), RxNorm for medications (InferRxNorm) and SNOMED CT for medical concepts (InferSNOMEDCT). Codes are what billing, insurance and research systems understand.

It is a pre-trained model: no clinical data to label, no model to train. It works on English text.

#HIPAA eligible versus HIPAA compliant

Comprehend Medical (and Transcribe Medical) are HIPAA eligible: AWS has the controls needed for the service to be used with PHI under a business associate agreement (BAA). That does not make your application compliant by itself. Compliance is shared responsibility: you sign the BAA, encrypt data at rest with KMS and in transit with TLS, restrict access with IAM, keep logs with CloudTrail and decide who may see what. Exam questions sometimes say “HIPAA compliant medical text analysis”; the answer is still Comprehend Medical, but know the precise term.

#Feeding Comprehend Medical

Store documents in S3 and analyse them (in real time per document or with batch jobs), analyse real-time data arriving through Data Firehose (lesson 13.6), or use Transcribe Medical to turn a doctor’s dictation or patient conversation into text that Comprehend Medical then analyses.

Clinical notes at a Chattogram hospital

Doctors dictate discharge summaries. Transcribe Medical turns them into text, and Comprehend Medical pulls out medicines, doses, conditions and the patient’s protected health information. Compare it with plain Comprehend on the same note.

Scenario

From free text to structured medical facts, with relationships between them.

Doctordictation
Transcribe Medical
Notes bucketKMS encrypted
Comprehend Medical
Comprehendgeneral NLP
Patient records
audio
Step 1 of 4
1

Dr Karim dictates: “Patient with type 2 diabetes, discharged on metformin 500 mg twice daily.” Transcribe Medical is the speech-to-text variant trained on medical vocabulary, so drug names come out spelled right.

#What goes wrong

  • Lambda times out waiting for an AI job: long audio, stored video and long Polly texts are asynchronous jobs. Start the job, return, and continue on the completion event (EventBridge for Transcribe, SNS for Rekognition Video and Polly tasks). Lambda stops at 15 minutes.
  • Offensive images get through, or harmless ones are blocked: the minimum confidence is too high (misses) or the block line too low (false blocks). Use two thresholds and send the grey zone to A2I.
  • A face is matched to the wrong person: the face match threshold is too low, or a decision about people was fully automated. Raise the threshold and add a human check; remember similarity is not identity.
  • Brand names are transcribed wrongly: add a custom vocabulary (or a custom language model for a whole domain).
  • A transcription job with redaction fails: the language does not support redaction (Bengali, for example). Check the language table; protect the unredacted transcript another way.
  • Polly rejects a request: the text is over 3,000 billed characters (use StartSpeechSynthesisTask), or an SSML tag is not supported by the chosen engine (emphasis or whispering on a neural voice).
  • The same acronym is mispronounced in every article: use a lexicon, not SSML in every article.
  • A brand name gets translated: add custom terminology.
  • Comprehend returns an error for a Bengali review: the language is not supported for that feature; translate first.
  • A Mixed review is treated as Neutral: the code only checks Positive and Negative. Handle all four classes.
  • The bot guesses wrong instead of handing over: use the fallback intent and a sensible confidence threshold, and transfer to an agent.
  • The same S3 object is processed twice: S3 event notifications are delivered at least once. Make the handler idempotent (for example a conditional write in DynamoDB keyed on the object key and version) so a duplicate event does not bill twice or publish twice (lesson 18.2 covers the full event-driven pattern).
  • Someone builds a custom model for a common task: a SageMaker AI project to detect cars in photos or the sentiment of reviews is over-engineering; the pre-trained service is cheaper, faster to ship and needs no ML team.

#AI services in system design

#Chaining services

The services are building blocks; real systems chain them, usually with S3, Lambda and EventBridge or Step Functions in between (lesson 15.7):

  • Call analytics: Transcribe (with redaction) → Comprehend (sentiment, entities) → results in S3 → Athena and QuickSight dashboards (lessons 17.1 and 17.5).
  • Multilingual voice: Transcribe (Spanish audio) → Translate (to English) → Polly (English speech), for example to dub a video or help an agent understand a caller.
  • Bengali reviews: Translate (bn → en) → Comprehend.
  • Clinical documentation: Transcribe Medical → Comprehend Medical → patient record.
  • Media library: Rekognition labels and faces + Transcribe for the speech → indexed in OpenSearch for search (lesson 17.3).
  • Contact centre: Connect + Lex for the IVR, Polly for the voice, Transcribe Call Analytics for the recordings.

#The event-driven pattern

Exam questions describe processing flows: user uploads a file → S3 bucket → S3 event notification (or EventBridge) → Lambda → AI service → result stored in DynamoDB or S3. Short calls (one image to Rekognition, one review to Comprehend) can run inside the function; long ones (audio, video, long texts) are asynchronous jobs whose completion triggers the next step. The full pattern, with Textract and the idempotency details, is in lesson 18.2.

Using managed pre-trained services instead of building and running your own models is an architectural best practice: less to operate, faster to deliver, pay per use. It fits the operational excellence and cost optimisation pillars of the Well-Architected Framework (chapter 26).

#Security and privacy

  • Encryption: these services encrypt data in transit with TLS; data you keep (S3 buckets, transcripts, custom vocabularies, face collections) should be encrypted with KMS (chapter 21), and several services accept a KMS key for their output.
  • Access: IAM policies control who can call which API (rekognition:SearchFacesByImage, comprehendmedical:DetectPHI), and services that read your S3 objects need permission to read them.
  • Private network: most of these APIs can be reached through VPC interface endpoints (PrivateLink), so calls from private subnets never cross the internet.
  • Data use: AWS may use content processed by some AI services to improve them unless you opt out through an AWS Organizations AI services opt-out policy. Regulated workloads usually opt out.
  • Audit: API calls are recorded in CloudTrail.

#Picking the right service

Use the picker below to practise the exam’s core skill: from what you have and what you want, name the service, and know why the look-alike is wrong.

AI service picker

Choose what you have, then what you want done with it. The picker names the service, says why, and shows the look-alike that the exam uses as a trap.

1. What you have

A news portal receives thousands of reader photos and short videos a day.

2. What you want done

Choose a task to see the answer.

#Try it yourself

These commands use the AWS CLI v2 in ap-south-1 (Mumbai). Bucket names, account ID 111122223333 and all data are placeholders. Every call is billed per request, minute or character, so the costs are tiny, but delete what you create.

# 1. Rekognition: labels and moderation on a photo in S3
aws rekognition detect-labels --max-labels 5 --min-confidence 80 \
  --image '{"S3Object":{"Bucket":"news-photos-example","Name":"2026/11/mirpur-flood.jpg"}}' \
  --query 'Labels[].[Name,Confidence]' --output text
# Flood   97.1
# Street  95.4
# Person  99.2
# Car     88.0
# Water   96.3

aws rekognition detect-moderation-labels --min-confidence 60 \
  --image '{"S3Object":{"Bucket":"news-photos-example","Name":"comments/beach.jpg"}}'
# { "ModerationLabels": [ { "Confidence": 71.4, "Name": "Suggestive", "ParentName": "",
#     "TaxonomyLevel": 1 } ], "ModerationModelVersion": "7.0" }
# 71% is between 60 and 90: send to human review (A2I), do not auto-block.

# 2. Rekognition: a face collection and a search
aws rekognition create-collection --collection-id staff-faces
aws rekognition index-faces --collection-id staff-faces --external-image-id R-17 \
  --image '{"S3Object":{"Bucket":"news-photos-example","Name":"staff/r17.jpg"}}' \
  --query 'FaceRecords[].Face.FaceId'
# [ "5f4d1a2b-0c3e-4b5a-9d8e-7a6b5c4d3e2f" ]
aws rekognition search-faces-by-image --collection-id staff-faces --face-match-threshold 95 \
  --image '{"S3Object":{"Bucket":"news-photos-example","Name":"2026/11/press-conf.jpg"}}' \
  --query 'FaceMatches[].[Face.ExternalImageId,Similarity]' --output text
# R-17    99.21

# 3. Transcribe: a call with speaker labels and PII redaction, plus a custom vocabulary
aws transcribe create-vocabulary --vocabulary-name wallet-terms --language-code en-US \
  --phrases bKash Pay-Bill Cash-Out
aws transcribe start-transcription-job --transcription-job-name call-20261101-0001 \
  --language-code en-US --media MediaFileUri=s3://wallet-calls-example/2026/11/01/0001.wav \
  --output-bucket-name wallet-transcripts-example \
  --settings ShowSpeakerLabels=true,MaxSpeakerLabels=2,VocabularyName=wallet-terms \
  --content-redaction RedactionType=PII,RedactionOutput=redacted
aws transcribe get-transcription-job --transcription-job-name call-20261101-0001 \
  --query 'TranscriptionJob.TranscriptionJobStatus'
# "COMPLETED"   (the transcript JSON is in wallet-transcripts-example)
# In the transcript: "spk_1: my card number is [PII] and it was charged twice"

# 4. Polly: a lexicon, then SSML with a pause, then a long article as a task
aws polly put-lexicon --name newsroom --content file://newsroom.pls
#   newsroom.pls maps BUET to the alias "Bwet" and AWS to "Amazon Web Services"
aws polly synthesize-speech --engine neural --voice-id Kajal --language-code en-IN \
  --lexicon-names newsroom --text-type ssml \
  --text '<speak>Breaking news.<break time="1s"/> BUET students win the robotics final.</speak>' \
  --output-format mp3 headline.mp3
# { "ContentType": "audio/mpeg", "RequestCharacters": 50 }
aws polly start-speech-synthesis-task --engine long-form --voice-id Ruth \
  --text file://sunday-feature.txt --output-format mp3 \
  --output-s3-bucket-name news-audio-example --sns-topic-arn arn:aws:sns:ap-south-1:111122223333:polly-done
# { "SynthesisTask": { "TaskStatus": "scheduled", "RequestCharacters": 18234, ... } }
# (check that the chosen engine and voice are offered in your Region)

# 5. Translate with custom terminology
aws translate import-terminology --name shop-brands --merge-strategy OVERWRITE \
  --terminology-data Format=CSV,File=fileb://brands.csv
aws translate translate-text --source-language-code en --target-language-code bn \
  --terminology-names shop-brands --text "Apex leather shoes, cash on delivery"
# { "TranslatedText": "Apex চামড়ার জুতা, ক্যাশ অন ডেলিভারি", "SourceLanguageCode": "en",
#   "TargetLanguageCode": "bn", "AppliedTerminologies": [ { "Name": "shop-brands", ... } ] }

# 6. Comprehend: sentiment (including Mixed), entities and PII
aws comprehend detect-sentiment --language-code en \
  --text "The camera is excellent, but the battery dies by lunchtime and the seller never replied."
# { "Sentiment": "MIXED", "SentimentScore": { "Positive": 0.09, "Negative": 0.08,
#   "Neutral": 0.02, "Mixed": 0.81 } }
aws comprehend detect-entities --language-code en \
  --text "I ordered a Walton fridge from Daraz on 3 March and it reached Sylhet in two days." \
  --query 'Entities[].[Type,Text]' --output text
# COMMERCIAL_ITEM  Walton fridge
# ORGANIZATION     Daraz
# DATE             3 March
# LOCATION         Sylhet
# QUANTITY         two days
aws comprehend detect-pii-entities --language-code en \
  --text "Call me on 01700-000000, account 0000-1111-2222." \
  --query 'Entities[].[Type,Score]' --output text
# PHONE                0.9998
# BANK_ACCOUNT_NUMBER  0.9991

# 7. Comprehend Medical: entities with dosage, and PHI
aws comprehendmedical detect-entities-v2 \
  --text "Patient with type 2 diabetes, discharged on metformin 500 mg twice daily." \
  --query 'Entities[].[Category,Text,Attributes[].Type]'
# [ ["MEDICAL_CONDITION", "type 2 diabetes", []],
#   ["MEDICATION", "metformin", ["DOSAGE", "FREQUENCY"]] ]
aws comprehendmedical detect-phi \
  --text "Rahima Begum, 54, of Agrabad, hospital no. CTG-0042, admitted 28 Oct 2026." \
  --query 'Entities[].[Type,Text]' --output text
# NAME     Rahima Begum
# AGE      54
# ADDRESS  Agrabad
# ID       CTG-0042
# DATE     28 Oct 2026

# 8. Lex V2: send text to a published bot (IDs are placeholders)
aws lexv2-runtime recognize-text --bot-id ABCDE12345 --bot-alias-id TSTALIASID \
  --locale-id en_US --session-id caller-0001 --text "I want a 30 GB pack for a month" \
  --query '[sessionState.intent.name,sessionState.intent.state,messages[0].content]'
# [ "BuyDataPack", "Fulfilled", "Your 30 GB pack is active for 30 days." ]

# 9. Clean up
aws rekognition delete-collection --collection-id staff-faces
aws transcribe delete-vocabulary --vocabulary-name wallet-terms
aws polly delete-lexicon --name newsroom
aws translate delete-terminology --name shop-brands

Also try in the console: the Rekognition demo page (drop in your own photo), the Comprehend real-time analysis page (paste a mixed review), and the Polly console (type a sentence, switch between standard and neural voices, then add a <break>).

#Summary

Key takeaways

  • Pre-trained AI services: AWS trained the model; you only call it (inference) through the SDK, CLI or API. No ML expertise, no training data, no servers; pay per image, minute, character or request. The exam prefers them over a custom SageMaker AI model whenever they fit.
  • Rekognition: images and video. Labels, text in photos, face detection and analysis (estimates, not facts), face search against a collection with a match threshold, celebrities, PPE, content moderation with a minimum confidence, and human review of the grey zone in A2I (closed to new customers since July 2026, still the exam’s answer). Custom Labels for your own objects. Stored video is asynchronous with SNS; people pathing is discontinued, streaming video closed to new customers.
  • Transcribe: speech → text (ASR). Speaker diarization, custom vocabulary, custom language models, PII redaction ([PII], supported languages only), automatic language identification, batch (async, EventBridge) or streaming; Call Analytics; Transcribe Medical.
  • Polly: text → speech. Standard, neural, long-form, generative engines. Lexicon = how a word is pronounced everywhere (acronyms, stylised words); SSML = how one text is read (pauses, emphasis, phonemes, breathing, whispering, newscaster; some tags only on some engines). 3,000 characters per call, async tasks to S3 for long texts; speech marks for highlighting and lip-sync.
  • Translate: neural machine translation, real time or batch from S3; custom terminology keeps brand names; localisation of sites and user content.
  • Lex = chatbots and voice bots (ASR + NLU, Alexa technology): intents, slots, Lambda fulfilment, fallback intent; web, mobile, Slack, not only phones. Connect = cloud contact centre, omnichannel, pay per use. IVR with conversational AI = Connect + Lex: call → Connect → Lex → Lambda → CRM.
  • Comprehend: NLP for any text: language, entities, key phrases, sentiment (Positive, Negative, Neutral, Mixed), syntax, PII, custom classification and entities. Topic modelling is closed to new customers. Limited languages: translate Bengali first.
  • Comprehend Medical: clinical text → medications with dosage, conditions, tests, PHI (DetectPHI), ICD-10-CM and RxNorm codes. HIPAA eligible; compliance is shared responsibility. Fed from S3, Firehose or Transcribe Medical.