I assume these are electrolytic capacitors,
the normal rule of thumb is use 'at least' twice the voltage rating for the capacitor as the capacitor will EVER have applied across its terminals, & for safety/reliability even more is better (overbuild).
So fundamentally - 3.7 volts x #S battery size times 2, 3 or more!
ex: 6s - 6 x 3.7 volts = 22.2 volts x 2 = 44.4 volt capacitor with rating OR higher! - this is true for ANY size/type capacitor small or large.
So really, the OP's 6s battery usage is actually NOT safe if the caps are 35v !! (as in many things you can get by for a while before failure occurs or caps are better than labeled !!)
And also, I am talking CONSTANT DC voltage, but if you have 'noisy' voltage with spikes above the norm battery DC voltage, then THIS should be the voltage used for the capacitor NOT the steady DC voltage - thus 3x or 4x is MUCH safer - if you care for ultimate reliability.
This 'noise' is 'induced' voltage into wires from some electrical noisy producing device - likely the ESC itself via radiation/induced into the battery lead wires (shielded wires could be used - both for input to ESC and its output)
FYI: I am sure Amazon will overbuild their delivery copters in most/all ways if they really desire reliability !
Since it seems ALL ESC's are multi-voltage input (multi battery sizes can be used), then the ESC should be 'engineered/built' to the highest voltage that can be applied times 2 or more (x3 or x4), or it is poorly built and can really fail.
Any/all capacitors are obviously imperfect, meaning they have small imperfections throughout their layers and surfaces, it is when a 'hole/imperfection' occurs that allows current to flow too fast which will then cause their destruction - too much current flow (temp increases) and can literally explode - the usual cause of failure is too much voltage across it or age.
A capacitor acts much like a battery, but it only acts to store/maintain voltage - very little current will/should flow, a battery stores both voltage and current - which is power or amp/hours @ some voltage = watts.
So a capacitor is 'trying' to maintain a voltage, but its voltage 'storage' will be 'drained/released' quickly depending on its 'load' ie energy sucked from it by ESC which is what OP is doing - supplying a given voltage 'longer' before decreasing - making a more perfect battery as far as ESC is concerned!
FYI: Audiophiles have worked for VERY many years, trying to improve their music sources, pre-amps and power amplifiers by trying to 'stabilize' the power/voltage throughout their systems - similar to the OP's effort, the diff is that they can spend money for big heavy capacitors and huge 120v line filters/regulators (electronic and Inductors/capacitors) in order to have 'pristine' spike free constant voltage/power allowing the system to delivery ONLY an 'amplified' signal and nothing extra OR less - from the music source to the speakers - I have personally seen $200,000 sound systems (2 speakers!). Audiophiles use shielded wire/cables for EVERYTHING if possible - no stray voltage - perfect voltage & 99.99999999999 % pure copper wires/cabling !! (not that most of us could hear it really - no mp3's here !!!! )