AI Audio and Video
Generative AI can now speak, sing and make short videos, which opens many uses and some serious risks.
AI Audio: Four Main Types
| Type | What it does | Example use |
|---|---|---|
| Text-to-speech (TTS) | Turns written text into natural speech | Voice-over for a YouTube tutorial |
| Speech-to-text (STT) | Turns spoken words into text | Meeting notes, subtitles |
| Voice cloning | Copies a specific person's voice | Dubbing your own videos into another language |
| Music generation | Creates music from a description | Background music for a reel |
Many chat assistants also have a voice mode, where you speak and the AI answers aloud in real time.
How AI Speech Works (Simply)
Sound is a wave. A computer stores it as thousands of numbers per second (samples). A speech model learns how text maps to these sound patterns from many hours of recorded speech with transcripts.
This small program shows that audio is just numbers. It makes one second of a simple tone:
Example
import math
sample_rate = 8000 # numbers per second
frequency = 440 # the musical note A
samples = [math.sin(2 * math.pi * frequency * t / sample_rate)
for t in range(sample_rate)]
print("Number of samples in 1 second:", len(samples))
print("First 3 samples:", [round(s, 3) for s in samples[:3]])Number of samples in 1 second: 8000 First 3 samples: [0.0, 0.339, 0.637]
A speech model generates numbers like these, but in the patterns of a human voice.
AI Video
Text-to-video models create short clips from a prompt, and image-to-video models animate a still picture. They use ideas similar to image diffusion, but they must also keep things consistent from frame to frame: a person's face and clothes should not change between seconds.
A clear video prompt describes the scene, the movement and the camera:
Example
A slow camera pan across a busy Karachi street market at night,
colourful lights, people walking, light rain, cinematic style,
8 seconds, no text on screen.Other video features include automatic subtitles, removing backgrounds, AI avatars that read a script, and dubbing a video into Urdu or English while matching the lip movements.
Practical Uses
- Teachers turning notes into narrated lessons
- Small businesses making product reels without a studio
- Freelancers offering voice-overs, subtitles and dubbing
- Making content accessible: captions for deaf viewers and audio for people who cannot read easily
Limits
- Videos are usually short, and details can glitch (hands, text, fast motion).
- Urdu and regional voices are improving but can still sound less natural than English.
- Generating video costs much more computing power than text.
Deepfakes: The Big Risk
A deepfake is fake audio or video that makes a real person seem to say or do something they never did. Deepfakes have been used for scams, fake political clips and harassment.
Only clone your own voice, or a voice you have clear written permission to use. Never make fake media of real people to mislead or embarrass them.
To protect yourself:
- Be careful of urgent voice notes or calls asking for money, even if the voice sounds familiar.
- Check the source of shocking videos before sharing.
- Look for labels: many platforms now mark AI-generated content.
Exercise
Exercise
What does a text-to-speech (TTS) tool do?