The former Frasier star and his partner welcomed their second child together on Tuesday, 22 July in Los Angeles.
In a statement to ET Online, the actor said: "We are blessed and excited to have this lovely young man join our family. He is magnificent."
The baby's full name is Kelsey Gabriel Elias Grammer, but he'll go by the name Gabriel as it's tradition in the Grammer family for people to use their middle name.
Kelsey, 59, and Kayte, 32, are also parents to daughter Faith, two.
In addition, Kelsey has four children form previous relationships: Spencer, 30, Greer, 22, Mason, 12, and nine-year-old Jude.
Congrats to the new parents!
Got an iPad or iPhone? Reveal Mag is specially designed to fit!
Stars who welcomed babies in 2014 in pictures:
Copyright: Rex Features WENN/Matthew Taylor/Daniel Deme/Josiah True/Matthre Taylor/Joe