The British-born star was a producer on the long-running US talent show, but told fans that bosses felt the programme needed a new direction as its enters its 13th season.
Nigel tweeted: "Just had 10 days in the Bahamas, rain everyday. I get back to the States and get fired. Sad!"
He them assured fans that the decision wasn't personal, saying: "They just feel IDOL needs new leadership after 12 seasons."
Nigel will remain as head judge on So You Think You Can Dance, another talent show that airs on the Fox network in the US.
Ex-Idol contestant Jacob Lusk was just one of many tweeting their sadness that Nigel was leaving, saying: "You are one the best things that's ever happened to TV."
Fox confirmed Nigel's departure - and that of fellow producer Ken Warwick - thanking both for their immense contribution to the show, which has discovered stars including Kelly Clarkson, Jennifer Hudson, Daughtry, Adam Lambert and Carrie Underwood.
The double exits mean it will definitely be all change at Idol next season: judges Mariah Carey, Nicki Minaj and Randy Jackson have also confirmed they are leaving, meaning as it stands, host Ryan Seacrest and fourth judge Keith Urban will be the only familiar elements when the show returns.
> Mariah Carey poses for sweet picture with her daughter and mother
> Jessie J impresses Nicki Minaj with American Idol performance