One thing I’ve been thinking about in this area is you could make the “plugin system” completely compiled. Users would have some toml or .rs file they would change and point the rust compiler at a directory and come out with a new binary. You could still add in a “Lua plugin” plugin which enables lua-based plugins without forcing all users to pull down the gear required to use that in rust. It would also make it possible for people to create various “distributions” of the editor with varying complexity and size depending on user needs

I'd say something like LUA or python so people could easily write their own plug-ins. If speed really is a concern then I'd go with wasm.

I’ve been thinking about exactly this for a long time. My only thoughts are you should check out helix to see what you can learn from them if you haven’t already, and you should post or dm me the repo so I can see if there’s anything I can help with!

One way to implement Rust-based "plugins" is to turn the whole application/plugins relation on its head: Publish the main application as a crate, which lets the user build their own executable that uses the main application as a library. That way they can customize the app however they like, if the main app just has hooks for it.

Just in case you weren't aware, there is an emacs-inspired editor in rust: https://github.com/zee-editor/zee

Not sure how active development is anymore. Maybe it could be useful to you for ideas/inspiration.

I think helix has been talking a lot about this issue too. Their solution, as far as I understand it, is to use a LISP language for scripting and extensibility and a wasm plugin system for performance critical plugins. Not sure how far it is into implementation, but something to look out for.

It's not exactly what you are looking for, but you could check emacs-ng. It tries to replace some of emacs' internals with rust and expand the possible scripting lenguajes that can be used with it.