Augustus is responsible for rebuilding Rome and did a lot of good things for the empire, but he also forced everyone in Rome to agree with him with a price of brutality if they did not, and he removed all of the Romans' rights and freedoms. The large, expanded government that Augustus built was also difficult to sustain moving forward because it required complicit approval.
After Augustus' death, the Roman Empire faced many problems, including terror tactics and assassination. He was also harsh with his family and banished his daughter due to her open sexuality, which was against his conservative beliefs. He did not put others first and only thought in terms of his own gain.