I would start by eliminating the Bureau of Indian Affairs.
Let's face it - if the "native people" are truly sovereign nations, they need to be fully treated as such.
We need to recognize them in the United Nations - and then have relations with them fall under the State Department.
Next - I would do away with National Parks, Forests, and the Bureau of Land Management.
The Constitution is very specific in regards to land owned by the federal government - and none of those are covered.
Control should be given to the states those lands are within.
Next - I would do away with every court system that is not under the Judicial Branch.
In every single one of those courts, the defendant is treated as guilty until proven innocent.
The biggest example of this abuse of power, is the IRS. You have to go through their court system before you are allowed to defend yourself in the Judicial Branch. During that time, you are likely to spend so much money fighting the government that you can't afford to take the case to the Judicial Branch level.
Congress is responsible for a lot of unnecessary "drama" that wastes taxes.
All we have to do is look at all the committees and hearings they have, which are focused on politics rather than actual work.