xfwm4 could work w/o Xfce, though I doubt that it would be worth the effort to script the missing bits by hand. Xfce is pretty modular; once you turn off the tracker/indexer, and whatever useless package manager gui the distro may have included (e.g., 'dnfdragora'), it's pretty lightweight. You can also turn off the compositor. The stock xfce4-panel is also miles ahead (IMHO) of various independent panel programs, both in functionality, as well as looks -- and its widgets are also entirely modular.

labwc is a window manager in the vein of openbox; I guess under wayland a window manager has to be a compositor too (?); but it's no different from sway in this regard.

There's also wayfire; which is a bit more beefy, and aims to preserve all the compiz plugins. Some of those are notorious for being silly eye candy (windows that burn down on close; wobbly windows, etc.) but others are pretty useful (esp. those that emulate the exposé view from OS X; pinning/grouping windows, etc.) -- though in my experience it isn't as stable as labwc; which is understandable because it's a lot more complex.