Apparently there's a bunch of low key bike races on gravel roads in the midwest called 'Gravel Grinders'. One is called the 'Heck of the North' which is a play on the 'Hell of the North' which is the nickname for the Paris-Roubaix cobblestone race in northern France.
There is even an emerging class of bicycle for this type of riding. Existing cross bikes and touring bikes would work perfectly, but these specialized rides tend to focus on drop bars, wide tires and steel frames, perfect for driving your rig over unpaved roads and soaking up the bumps. Many have disc brakes.