Spencer, who has had numerous relationships on MIC, made the statement while sitting next to his ex-girlfriend Louise Thompson on Daybreak.
When asked if it "gets confusing" because so many of the cast are romantically linked, Spencer grinned and said: "It can be...err...a bit incestuous!"
Spencer added: "The cast is ever growing, so I'm sure that we will always have room for which to expand.
"It can be a bit weird spending time with exes and people you wouldn't necessarily spend time with, but that just fuels the drama."
Lorraine Kelly also asked it the E4 show is true to life and he replied: "I wouldn't say it's true to life, but in order to create an entertaining programme, I think perhaps you have to focus on things that might be aspirational to certain people.
"We don't just roll around in Bentleys and drink champagne all day, otherwise we'd be fairly unproductive."
Last month it was revealed that there is to be a Made In Chelsea New York spin-off and, although they don't yet know which cast members are going, Cheska Hull has her hopes up!
Cheska, who was also on the ITV1 breakfast show, said: "We're going from May to July to film a mini series of six episodes and all three of us are desperate to go!"
Louise added: "It'll makes all the blood, sweat and tears worthwhile!"
Blood, sweat and tears...? Err, tears maybe...
Tonight, Made In Chelsea returns to E4 for the seventh series at 10pm.
Watch the trailer here: