My question might be silly, but I planning to install my server to a USB drive. The USB is way slower than a HDD/SSD, so will it influence the performance of the OS? Or the OS is mainly running in RAM?
It works well to run Ubuntu and Ubuntu family operating systems via USB. Particularly the USB 3 interface is fast enough.
In most USB 3 pendrives the memory cells are limiiting the speed, so it is a good idea to get a pendrive with specified and high speed, or even better use a fast SSD in an external USB 3 and/or eSATA box. An SSD has memory cells with longer lifetime (can manage more write cycles, before it wears out).
See the following links and links from them for more details.
With a server, well there will probably be a lot of write cycles ....
If you intend to install a server, I think the lifetime of USB pendrive flash memory will be too short, because you can expect a lot of write cycles. But a HDD or SSD connected via USB 3 will work well.
The short answer to your question is yes you can do it and yes the OS will likely run slower. However, many people are running TOR strictly from a thumbdrive for security reasons. I wouldn't advise running server apps this way but as a simple desktop/laptop environment, you can achieve acceptable performance doing what you propose.