38

I have a small homelab running a few services, some written by myself for small tasks - so the load is basically just me a few times a day.

Now, I'm a Java developer during the day, so I'm relatively productive with it and used some of these apps as learning opportunities (balls to my own wall overengineering to try out a new framework or something).

Problem is, each app uses something like 200mb of memory while doing next to nothing. That seems excessive. Native images dropped that to ~70mb, but that needs a bunch of resources to build.

So my question is, what is you go-to for such cases?

My current candidates are Python/FastAPI, Rust and Elixir, but I'm open for anything at this point - even if it's just for learning new languages.

top 33 comments
sorted by: hot top controversial new old
[-] tyler@programming.dev 1 points 5 days ago

Have you tried tuning your jvm settings? 200mb is a lot for a simple app. You can get that pretty dang low just by proper tuning of jvm settings.

[-] lilja@lemmy.ml 26 points 1 week ago

If you want something with a small footprint I would personally go for Rust, but anything that compiles to a static binary is going to be better than something that needs a dedicated runtime.

Python is what I use for small one-time scripts and utility stuff that doesn't need to run long, but it may be worse than Java..

[-] lambalicious@lemmy.sdf.org 18 points 1 week ago

Languages

C.

Frameworks

C.

That said, Python and Rust are great for setting up "starting up" / "small task" apps and growing up from there.

[-] leisesprecher@feddit.org 9 points 1 week ago

There's nothing to really grow. It's mostly just small helpers. Aggregate sensor data, pull data from A and push it to B every hour, a small dashboard, etc.

C is too involved for my case , I want to be productive after all.

Rust is already rather low level, though there are some cool looking frameworks.

[-] xmunk@sh.itjust.works 5 points 1 week ago

C is an extremely expressive language. There's a reason it won't die and, while we all love to shit on it for the memes, you can write perfectly safe software in it.

[-] leisesprecher@feddit.org 14 points 1 week ago

Of course, but I'm not productive in it.

If I have to do everything myself, it will take more time to get it done. The trade-off is of course always control/speed vs convenience, but C is definitely too inconvenient for me.

[-] xmunk@sh.itjust.works 3 points 1 week ago

That's fair enough.

[-] mysteryname101@lemmy.world 2 points 1 week ago

I’d lean towards C# with AOT compilation if you’re not using reflection, you should be ok.

[-] lambalicious@lemmy.sdf.org 2 points 1 week ago

Yeah definitively sounds like even more support for Rust and/or Python in this sense.

[-] FizzyOrange@programming.dev 8 points 1 week ago

you can write perfectly safe software in it.

In the same way that you can safely walk through a minefield.

I dunno what you mean about it being an expressive language either. I would say it is relatively low on the expressiveness scale compared to something like Python or OCaml. It's basically as expensive as Go which is renowned for being unexpressive. Maybe you didn't mean "expressive".

[-] Ephera@lemmy.ml 3 points 1 week ago

If you've so far been able to do this stuff in Java, then presumably all your hardware has an OS and such and you don't need this, but a colleague has been having a lot of fun with Rust and proper embedded development.

It's very different from regular development, as you've got no OS, no filesystem, no memory allocations, no logging. It can most definitely be a pain.
But the guy always has the biggest grin on his face when he tells that he made a custom implementation of the CAN protocol (TCP is too complex for embedded 🙃) and that he had to integrate an LED to output error information and stuff like that. At the very least, it seems to be a lot less abstract, as you're working directly with the hardware.

[-] beeng@discuss.tchncs.de 16 points 1 week ago* (last edited 1 week ago)

Go, you get small static binary, easy to code, and good performance.

[-] rwdf@lemmy.world 5 points 1 week ago

Second this, seems like a great use case for Go.

[-] MajorHavoc@programming.dev 5 points 1 week ago

Ditto. I use Go for this kind of thing.

[-] purplepudding@sopuli.xyz 14 points 1 week ago

Go is quite nice for this, generally low traffic services are less than 100mb used memory if you're using the standard lib stuff and it's not a huge jump from the JVM to Go.

[-] Greg@lemmy.ca 6 points 1 week ago

MicroPython is good for limited hardware projects.

[-] leisesprecher@feddit.org 2 points 1 week ago

Not that limited. Limited means an old thin client, not a microcontroller. I already set up a small web server on a pi pico with mpy, so it's quite impressive. But from what I understand, the interop with "MacroPython" is not that great.

Did you use mpy for x86 devices? Are the limitations worth it?

[-] ZILtoid1991@lemmy.world 5 points 1 week ago

The standard answer is C/C++.

The nonstandard answers are:

  • D. It's fairly close to Java, but with garbage collection you'll still at like 50MB depending on your usecase (it just doesn't immediately give the memory back to the OS for speed reasons), or you have to use some of the alt runtimes without a GC and many of the standard libraries.
  • Rust, if you can deal with immutability by default, a borrow checker, and many other quirks.
  • There's also languages like Zig, Nim, etc.
[-] leetnewb@beehaw.org 5 points 1 week ago

Other couple ideas to consider if job ambitions aren't a major thought:

Nim-lang / Mummy. Neat in being high level like python but compiled and can do low level stuff. Small ecosystem but good interoperability with c and Python. Can also compile to js. Target embedded to web, very flexible.

Also php. Some people say modern php looks more like java. Either way, lots faster than it used to be. Wildly productive language for web stuff. Laravel or Symfony frameworks.

[-] houseofleft@slrpnk.net 5 points 1 week ago* (last edited 1 week ago)

I feel like in a lot of ways, most languages are great candidates for this, for lots of different reasons!

  • Rust: Great choice because it produces a small, very well optimised binary. If you just care about the output binary being small and non-memory intensive, then this is probably a good call.

Buuuuut, Rust's compilation can be pretty resource intensive, so if you're actually developing on limited hardware:

  • C (or curveball option, Hare): produces a small, well optimised binary, with faster compilation. But less framework type things to help you on your way to apis/servers/etc.

Then there's the fact that it's a home server, so always on, meaning you actually have generous resources in some ways, because any available CPU is kinda just there to use so:

  • Python: has a runtime and can be pretty heavy CPU wise, but lots of frameworks, and in all honesty, would wind up being a lot faster to put stuff together in than Rust or C. Probably a great default option until you hit resource issues.

And then why not go whole hog into the world of experimental languages:

  • Roc: Doesn't have versions yet, so super new, but should produce a pretty small binary and give you higher level ergonomics than something like Rust or C, especially if you're into FP.

And then we're forgetting about:

  • Haskell: Haskell is the only true programming language, and any time there's a selection of programming languages, picking the one that isn't Haskell is the wrong choice. Just ask anyone who programs in Haskell.

But that doesn't factor in:

  • Javascript: Sooner or later, everything is just javascript anyway, why even try to resit?

Plus:

  • Assembly: Can you even trust that it's well optimised unless you're writing the assembly yourself?

Edit: My actual serious answer is that Rust + Rocket would be great fun if you're interested in learning something new, and you'd get very optimised code. If you just want it to use less memory that java and don't want to spend too much time learning new things then python is probably fine and very quick to learn. Go is a nice halfway point.

[-] PushButton@lemmy.world 4 points 1 week ago

Dotnet core, it's very similar to Java, it's portable and has a smaller footprint.

[-] Ephera@lemmy.ml 3 points 1 week ago

Hmm, I've never looked too much at benchmarks for this, but is there reason to believe Python would use less memory for a similarly complex project? It still needs a runtime, and it has to do a larger interpretation step at runtime (i.e. it needs to start from human-readable code rather than from bytecode)...

[-] leisesprecher@feddit.org 7 points 1 week ago

Python caches bytecode, so the translation happens only once.

Java loads everything immediately and keeps it in memory. All beans, all connections, etc. That takes up a ton of memory.

[-] norambna@programming.dev 2 points 1 week ago* (last edited 1 week ago)

Python / FastAPI will be better than Java in your situation and is easy to learn. Go should be even better and is also relatively easy to learn!

[-] milis@programming.dev 3 points 1 week ago

Go will be easy enough to get the job done, then you will have plenty of time rewrite it with whatever language you like.

That's at least my case. I have a cloud server that hosts a few services for news, weather, LLM client, etc.. Most of them, if not all, took me just 2-3 hours to get it up and running, and a couple hours more the following day to fine tune. Now I have a working service and I can move on to rewrite it in (in my case) Rust as a learning opportunity.

[-] benjhm@sopuli.xyz 3 points 1 week ago

Scala compiles either to native, js or jvm - obviously the IO / interface options vary between these envs, but the lang is the same. Recently Scala 3.5 incorporates a simple-to-use CLI which makes it easier to compile to native (or just run a small file as a script, or experiment with a repl), native binaries are small and fast, and there are some simple io libraries. Since you can also compile to jvm to interop with java, that might help with transition.

[-] leisesprecher@feddit.org 1 points 1 week ago

But that would mean either using Graal/native image or going full Scala, right?

I only used Scala for Gatling, where it's obviously very java-y.

[-] benjhm@sopuli.xyz 2 points 1 week ago

Indeed to use scala-native you'd need pure-scala libraries, but the core lib re-implements most java lib, and there are now small simple external libs available for common tasks like file management, database, etc. - for example check out the lihaoyi suite.
I mainly use scala-js (to make this) which was formerly a java app - as it compiles to both js and jvm (cross-project) can gradually convert stuff you already wrote. I've tried native for stuff like pre-processing data files.

[-] atomkarinca@lemmygrad.ml 3 points 1 week ago

Are you planning to compile the programs on the thin client? Although rust runs efficiently on a lot of hardware, compiling is gut-wrenching.

I have an rPi 1B running as a lightweight server and both rust and c++ applications take hours to compile (some of them take over a DAY). so, interpreted languages might be what you're looking for. my favorite is python. most distros have a lot of native packages in their repos. albeit a little weird to work on, perl is great, too.

[-] felsiq@lemmy.zip 3 points 1 week ago

To add to this, with rustup you can add different build targets than the current system - could let you build the binary on a more powerful pc and then just scp it over.

[-] Ephera@lemmy.ml 5 points 1 week ago* (last edited 1 week ago)

Yeah, we do this regularly at $DAYJOB, although we use Cross.

Basically, if you pull in any libraries with vendored C code, like e.g. OpenSSL, then you'd need to configure a linker and set up cross-compilation for C code.
Cross does the whole compilation in a container where this is already set up, so you just need to install Docker or Podman on your host system.

Basically:

cargo install cross

cross build --release --target=armv7-unknown-linux-gnueabihf

...and out plops your binary for a Raspberry Pi.

[-] atomkarinca@lemmygrad.ml 3 points 1 week ago

this is great.

[-] 31337@sh.itjust.works 3 points 1 week ago

Python is quite slow, so will use more CPU cycles than many other languages. If you're doing data-heavy stuff, it'll probably also use more RAM than, say C, where you can control types and memory layout of structs.

That being said, for services, I typically use FastAPI, because it's just so quick to develop stuff in Python. I don't do heavy stuff in Python; that's done by packages that wrap binaries complied from C, C++, Fortran, or CUDA. If I need tight-loops, I either entirely switch to a different language (Rust, lately), or I write a library and interact with it with ctypes.

this post was submitted on 08 Sep 2024
38 points (95.2% liked)

Programming

16971 readers
246 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev



founded 1 year ago
MODERATORS