Recently I started using my own self hosted search engine - LibreX...
When I went to add it as the default search engine for Firefox on the desktop - I found that the option to add custom search engines gone (replaced with a 'smarter' way??)
Here is the work around I found..
Go to the about:config page
Paste browser.urlbar.update2.engineAliasRefresh into the search area and set the value to True
Now when you go to the Settings page, you should see an Add button to add a custom search engine (like you used too...)
Lad