Luckily a passing cyclist spotted the frightened terrier, and phoned the emergency services.
They rowed up to the terrified terrier, and reaching over, they managed to free him.
Carefully they lifted the dog, which was shivering with cold, onto the boat.
They rowed the terrier back to safety, and it is now being cared for by the RSPCA, who are trying to find out who it belongs to.
A dog is recovering at an animal hospital after he tumbled down a steep canal and got trapped behind a ladder.
Luckily, a cyclist spotted the terrified terrier, and called the emergency services.
Officers from the fire and rescue service lowered a boat onto the canal and rowed up to the frightened animal, and managed to free him.
The dog was shivering with cold and had scuffed his nails trying to scramble out of the water.
RSPCA Animal Collection Officer, Boris Lasserre, said he hoped dog owners would learn a lesson from the rescue.
He said: "We would dearly like to reunite dog with owner.
"The dog was wearing a black collar, but sadly as he was not microchipped we have no idea whom he belongs to.
"Apart from his scuffed paws he is in good health." Mr Lasserre also thanked West Midlands Fire and Rescue service. He said: "We do rely on them during difficult rescues and this was one such rescue where a dog's life was in very real danger."
CBBC Newsround | Animals | Trapped terrier's amazing rescue