I think that my answer is generally unpopular with other Romanophiles, but I tend to feel that the Germanics of the middle to later periods (Goths, Huns, Vandals, etc.) were the biggest threat; essentially because it was the Germanics who pressured the Rhine and Danube borders almost continuously, eventually driving the proverbial nail into the coffin of the western empire. Understandably, the Germanics were a collection of many differing tribes, so perhaps one single tribe (ie the Marcomanni) may not have been as menacing as the entire Parthian empire, but I am referring to a collective Germanic people.
While I agree in part with your assessment on the status of Persian/Parthian "civilization" as compared to Rome, militarily the threat was local. There was an unquestionable long-standing rivalry between Rome and it's eastern neighbors for influence in "Asia", but the core of the Roman empire was never truly threatened. Roman gains (under the likes of Corbulo, Trajan, Septimius Severus, etc.) were never permanent, but nor did the destruction of Crassus or the failure of Macrinus for example result in permanent gains for Parthia. The eastern Roman empire maintained itself against its eastern neighbors long after the fall of the west.
(Also note that my quote is only regarding the "imperial" period, therefore conveniently discounting such Republican era threats as Carthage, Seleucia (Antiochus), Pontus (Mithridates), etc.)
I'm certainly not as well versed in this area as Primus, but Alessandro Barbero in his "The Day of the Barbarians", states: "The Persians had no wish to enter Roman territory and settle there; at most, they wanted to conquer the empire's rich eastern provinces. Here the clash was not between civilization and barbarians but between two civilizations that despised each other and had fought for centuries."
Barbero argues that in the mid-to-late 4th century AD "had an ambivalent attitude toward...barbarians." While the masses absolutely feared barbarians, the government looked at them as a resource to be used for war - sparing citizens who were increasingly turning away from careers in the army - to keep them working the land and generating revenue for the empire. "The barbarians were a potential resource that should not be wasted."
A few years before the Battle at Adrianople in 378, Emperor Valens allowed a mass immigration of barbarians across the Danube. In part, the barbarians would feed Valens growing army in the East - he was preparing for war against the Persians. In short, this immigration ultimately became an invasion, Valens abandoned his war against Persia and he was ultimately killed and his army was defeated at Adrianople.
So...that's all a long way of saying that the Empire's greatest enemy really probably depends on what timeframe you review. Could one argue that the Empire's greatest enemy was Caesar as he crossed the Rubicon? Clearly the Empire felt that Persia was a great threat (or perhaps it was considered an opportunity) in the 4th Century. And it's certainly hard to argue that the barbarians, as a whole, became the greatest enemy in the later years, and perhaps the most persistent throughout the Imperial Roman period.
"Internal enemies" are difficult to asess, because it is usually hard to define which side was more "Roman" than the other; even Caesar couldn't have done what he did without the support of a huge proportion of the Roman population. In any case, Mereoveo's original question was about "other peoples".
"Barbarians" is just the Roman term for "aliens" : all aliens.
We all know that the city-state that Rome was at the early IV century BC conquered anything from the Atlantic to the Euphrates; irrespectively of their intended historical or moral justifications, that simply can' be explained by mere coincidence or perpetual self-defense.
There are very few absolutes in History, but one of them is that up to the early II century AD Rome attacked and conquered (or at least tried to) absolutely all its neighbors, including of course 100% of its allies and friends; even the Parthians were technically Roman allies (of Sulla) against Tigranes the Great.
Plainly, Rome was permanently attacking, even at the Hannibalic War.
Then, strictly speaking, it was Rome which was initially (virtually by definition) the enemy of any barbarian (ie, non-Roman), even the unknown ones.
Rome became an extremely efficient military machine quite early in its history and there is no evidence that any enemy ever became strong enough not to be ultimately defeated by the undivided attention of the Legions.
When and where the expansion of the Empire stopped, it was entirely explained by logistic reasons; the Empire was just too big and complex to grow any further (ie, the phase II of Luttwak).
Even so, there were repeated Roman attacks well beyond essentially all the established borders when and wherever the Romans were (or feel) strong enough to try, from Caledonia to Arabia Felix, at least up to Julian (not to talk about the re-conquest campaigns of Justinian, Heraclius and others).
"Germanic people" is too unspecific; virtually anything west to the Rhine and north to the Danube (or Dacia at most) from at least six or seven centuries; people then which have quite few things in common among them, besides the fact of being "Barbarians".
Instead of "Germanic peoples", we might then select the Huns, The Visigoths or the Vandals.
As military menaces and irrespectively of the specific timeframe, I suppose we may measure them by the objective damage they actually did to the Empire.
My personal choice would then be the Fourth Crusade; however, the Islamic Arab armies of the VII century were probably an even more formidable enemy.
Selecting exclusively from the Germanic peoples of the V century, I think the Visigoths did the worst damage.
Thank you for your answer, very clearly and interest!