Official document: Captcha Library (anti bot obfuscated text string) CAPTCHA troubleshooting :
Formidable's CAPTCHA may not display because the font is not found.
You can try to load directly the page http://yourdomain.tld/typo3conf/ext/ameos_formidable/api/base/rdt_captcha/res/lib/cryptographp.inc.php to figure if there any error messages (this is the url of the image file output script).
The fonts used must be located in directory /typo3conf/ext/ameos_formidable/api/base/rdt_captcha/res/lib/fonts/
If the font's exention is uppercase (eg: .TTF) it will likely not work on UNIX systems. It is thus recomended to use lowercase extensions in your font files.
Also the /typo3conf/ext/ameos_formidable/api/base/rdt_captcha/res/lib/fonts/ directory + all the fonts contained inside must be chmoded to 775 for the CAPTCHA to work.