A number of companies now produce small "laptop in a box" PCs that can be used as small scale internal servers running Ubuntu, etc. A quick check reveals that the Intel NUC, Gigabyte BRIX, ASUS Vivo, and Zotac ZBOX are all pretty strong contenders.
These boxes all come with Intel i5 or i7 multi-core CPUs and can have up to 32GB of RAM and support M.2 and 2.5 inch SSD drives.
Since they are laptop CPUs and SSD drives, they make almost NO NOISE and require very little power! For an internal development environment, they have plenty of power. Most come with integrated 4K video cards and HDMI ports to connect a monitor.
I personally have 4 connected to a 4 port Keyboard/Video/Monitor switch that allows me to switch between them quickly. But they can be run as headless servers with ssh access from your main machine as well - much like you would do in AWS anyways - without the monthly bill.