The office of Emperor is all illusion. We call them that, they didn't. Mary Beard will insist "of course they were emperors!" which she has written in her book SPQR, but I disagree intensely. They did the same job - but 'Emperor' wasn't a proper job anyway (As discussed by Greg Woolf in interview with Mary Beard on tv).
The word 'emperor' descends from Imperator which meant 'Victorious General', a traditional honour given by soldiers to commanders who win wars, but later, used from Augustus onward to describe their status as de facto control over the legions - but then, legions were notoriously fickle and certainly not regiments in a state army organisation. All independent, and all prone to deciding loyalty for themselves. Please note that nearly half the major battles fought by Imperial Roman legions were against each other.
The thing is the Romans didn't organise their political and military sphere as we do today. They compartmentalised everything. So that when Augustus hands power back to the Senate, it isn't actually a ruse as is often described but a traditional requirement he obliges the Roman state with and one that only parts company with a certain allocation of power. He still has his powerful CV, his authority and status in Roman society, his senatorial influence, his wealth, his catalogue of influential friends and contacts, and most telling of all, the adoration of the common folk. He could afford to lose a privilege or two. It made his career less contentious.
Sorry, I'm getting distracted. Caligula. Well Claigula comes to power with the high hopes of Roman society but manages to alienate everyone except the remote common people within four years. He struggles to retain respect, and that black humour of his spoils everything. The elite tire of his antics and disrespect. But a few held grudges. Especially Cassius Chaerea, the Praetorian Prefect and war hero of the Germanic campaigns. Caligula ribbed him mercilessly for his soft voice, and for a man of action rewarded with the highest military post of the time, it was too much. Chaerea was among the leaders of the insurrection and helped murder Caligula in the passage leading from the theatre.