“What happens if the decayed tooth is not removed?”

Decay will progress, and if extensive enough can cause tooth to crumble and or cause the nerve to die.

Eventually the decay will progress and the tooth will rot below the gumline. The gum will then grow over top of the tooth and encapsulate it, thereby making it much more difficult to extract, not to mention causing pain and infection/swelling. A non-restorable tooth should be extracted promptly. Good luck.
Dr. Cyril Tahtadjian

All decayed teeth do not have to be removed. We save many teeth with decay. If it is treatable, it is best to keep it. If into the nerve and painful, we can still treat it with root canal therapy and post and core and crown. If the decay goes too far into the the tooth and toward or below the gum line, it must be removed even if no pain or you will get an access and bacteria into you bloodstream toward your heart or artificial joints. Follow your dentist's advice.

You don't mention the size of the decay in your question. Tooth decay - to use a simple analogy - is like rust on a car. You can wax or wash it as much as you want but unless you physically remove the rust (or the decay) eventually your muffler falls off, or your floor rots out, or worse.
Decay must be stopped to stop the progress of deterioration of the tooth. Otherwise symptoms of pain begin, infection can occur, and the tooth can cause general systemic problems from cellulitis (a severe infection which spreads to other parts of your body and even death if left untreated. I've seen tooth roots of teeth lost to decay after many years - which is rare and at first seems contrary to what I've just stated - become infected and patients hospitalized. So visit your dentist and get their professional advice to whether the tooth can be fixed or removed.

Couple terrible nights and then extraction.

The decay can affect the nerve, which then may cause an abscess. Infections can get worse.

Can cause dental/tooth abscess which can become a medical emergency

Assuming the decayed tooth is a non-restorable tooth (ex: hopeless diagnosis). If the decayed/fractured tooth is not removed, pain is certainly one of the potential side effects you might encounter. If the infection has already spread to the apex of the root, then infection can proliferate to the adjacent bone structure or other teeth/root. Please visit your dentist for a definitive diagnosis. Good luck.