"I will break the good news to our Islamic nation, which will... annoy the crusaders, and it is that the Shabab movement in Somalia has joined al-Qaeda," Zawahiri said in the video published on Thursday.
In the first part of the video, al-Shabab's leader Ahmed Abdi Godane, also known as Mukhtar Abu Zubair, addressed Zawahiri, saying: "We will move along with you as faithful soldiers."
Al-Shabab controls much of southern and central Somalia and has claimed responsibility for numerous kidnappings and bombings in the country.
The group said it was behind an attack on Wednesday that killed at least 11 people and wounded 34 others in the Somali capital.
Al Jazeera's Nazanine Moshiri, reporting from Nairobi in neighbouring Kenya, said that Mogadishu was "trying to rebuild itself after the al-Shabab withdrawal ... but the attack shows al-Shabab is able to get into the city and carry out attacks of this kind".
Al-Shabab, who are fighting to overthrow a fragile Western-backed transitional government in the war-torn Horn of Africa country, first proclaimed their allegiance to Osama bin Laden in a video distributed in 2009.
While counter-terrorism experts say al-Shabab has received advice and training from some members of al Qaeda, it has tended to see itself more as an ally or affiliate than a direct outpost of the core organisation.
Al-Shabab are facing increasing pressure from government forces and regional armies.
Armies from neighbouring countries are converging on them - Kenyan forces in the south, Ethiopian soldiers in the west, and an African Union force in Mogadishu made up of 10,000 troops from Uganda, Burundi and Djibouti.
A one-day conference in London to tackle the instability in Somalia and piracy off its shores is due to be held in two weeks time.