this post was submitted on 25 Sep 2026
58 points (91.4% liked)

Python

8060 readers
9 users here now

Welcome to the Python community on the programming.dev Lemmy instance!

๐Ÿ“… Events

PastNovember 2023

October 2023

July 2023

August 2023

September 2023

๐Ÿ Python project:
๐Ÿ’“ Python Community:
โœจ Python Ecosystem:
๐ŸŒŒ Fediverse
Communities
Projects
Feeds

founded 3 years ago
MODERATORS
 

A hard exercise to help build the right mental model for Python data.

The โ€œSolutionโ€ link visualizes execution and reveals whatโ€™s actually happening using ๐—บ๐—ฒ๐—บ๐—ผ๐—ฟ๐˜†_๐—ด๐—ฟ๐—ฎ๐—ฝ๐—ต: https://github.com/bterwijn/memory_graph

top 35 comments
sorted by: hot top controversial new old
[โ€“] farmgineer@nord.pub 6 points 6 days ago (2 children)

My brain threw a compile error on |=. I have no idea what that does without looking it up (I've basically never touched python having mostly worked in perl/shell back in the day and I rarely need scripting languages today). I'm also not sure what the pipe operator on its own is doing there.

[โ€“] bterwijn@programming.dev 5 points 5 days ago

If you are interested see the dictionary docs (https://docs.python.org/3/builtins/stdtypes.html#dict):

d |= other Update the dictionary d with keys and values from other, which may be either a mapping or an iterable of key/value pairs. The values of other take priority when d and other share keys. Added in version 3.9.

Pick your answer and check the Solution link.

[โ€“] syklemil@discuss.tchncs.de 5 points 6 days ago (1 children)

I didn't have previous exposure to that exact syntax either, but:

  • Python and plenty other languages have assignment variations with an operator like +
  • | in plenty of languages means logical OR
    • possibly bitwise, but that seems unlikely in Python
  • Dicts are sorta complex siblings of sets
  • In terms of sets, OR is equivalent to a union

Hence, a |=b should have the same meaning as a = a | b or a = a.union(b)

and then the quiz is about basically implementation details and what the actual semantics winds up being in a language like python, where stuff is mutable by default and references are implicit (even though the zen of python states a preference for being explicit)

[โ€“] bterwijn@programming.dev 3 points 6 days ago* (last edited 5 days ago) (1 children)

There is one important difference between a |= b and a = a | b, that is one part that makes this exercise difficult. See the dictionary docs (https://docs.python.org/3/builtins/stdtypes.html#dict):

d |= other Update the dictionary d with keys and values from other, which may be either a mapping or an iterable of key/value pairs. The values of other take priority when d and other share keys. Added in version 3.9.

Pick your answer and check the Solution link.

The zen of Python lies unless you're Dutch. I'm Dutch and when I look out of my office I see the CWI where Guido wrote the original Python versions back in the day, see "Python the documentory" on YouTube.

[โ€“] syklemil@discuss.tchncs.de 2 points 5 days ago (1 children)

I was trying not to spoil that a |= b and a = a | b have different semantics though, as that pretty much winds up giving away the answer IMO

[โ€“] bterwijn@programming.dev 1 points 5 days ago

Thanks, but I also give a solution link, so people can already cheat if they really want too.

[โ€“] apparia@discuss.tchncs.de 6 points 6 days ago (1 children)

Nice puzzle and tool. I picked the right answer but I wasn't overly confident.

You could list {1: [100]} as another possible answer, if you assume:

spoilerb |= {...} is equivalent to b = b | {...}, but you also realise the lists are never deep-cloned.

[โ€“] bterwijn@programming.dev 4 points 6 days ago

Thanks, maybe I should have added that possible answer instead of 'D', adding one more feels too many.

[โ€“] eleijeep@piefed.social 20 points 1 week ago (1 children)

The | and |= operators for dict were added in 3.9 which was released in 2020, so it's a relatively recent feature.

I like having an operator for set union but I don't like that a |= b has different semantics to a = a|b. This violates the principle of least surprise, since the former is supposed to be a shorthand for the latter.

[โ€“] bterwijn@programming.dev 6 points 1 week ago* (last edited 1 week ago) (1 children)

The difference between a |= b and a = a | b is exactly the same as the difference between a += b and a = a + b. Nothing new about that, try:

a = [1]
b = a
b += [2]
b.append(3)
b = b + [4]
b.append(5)

print(a)

or see: https://fosstodon.org/@bterwijn/116328991093782469

For immutable types there is no difference between a += b and a = a + b, try with a tuple.

[โ€“] sukhmel@programming.dev 7 points 6 days ago (1 children)

I feel that things like this lead people to say something along the lines of โ€˜mutability was a mistakeโ€™

I'm only half serious, but this is really surprising without Python background

[โ€“] bterwijn@programming.dev 3 points 6 days ago (2 children)

If you don't want mutability you have to go to a pure functional language like Haskell, but then you have to copy a big list every time you make a change. There are ways to optimize copying by secretly sharing data behind the scene but you pay a performance price in some way. Then again, Python is slow and has mutability but popular for other reasons.

[โ€“] sukhmel@programming.dev 4 points 6 days ago (1 children)

I don't think mutability is wrong as a concept, albeit I enjoyed learning and toying with Haskell, but I really think these functions in Python should have been two sets of operations, one set to do what a += b does and one set to do what a = a + b does.

I've been doing C++ for quite some time and amount of implicit things that happen magically and not everyone get them right and this leads to bugs and confusion had really grown on me with time, and I feel like this is the same pattern here, where we get implicit magic instead of being clear with intentions and results

[โ€“] bterwijn@programming.dev 3 points 6 days ago* (last edited 6 days ago) (1 children)

I think the confusion comes from a += b being equivalent to a = a + b for immutable types, so some people generalize that incorrectly to mutable types too. Otherwise I think it's pretty clear a += b mutates a, and a = a + b first computes a + b and then reassigns that to a so that its identity changes, just like in c = a + b.

If you implement these operations in a class you have to implement each dunder, __iadd__(self, other) and __add__(self, other) separately, same thing in C++.

[โ€“] sukhmel@programming.dev 1 points 6 days ago (1 children)

It is only clear if you already have a model of how labels work in Python and while it is understandable to me now, I wouldn't say this is anything I would expect

Then again, shadowing variables is allowed in other languages and is also a source of confusion and mistakes at times

[โ€“] bterwijn@programming.dev 1 points 5 days ago (1 children)

The point of the visualization at the Solution link is precisely to help people get the right model of how labels and the data model in general works in Python. See the Explanation link for more details.

[โ€“] sukhmel@programming.dev 1 points 5 days ago (1 children)

Yeah, the visualisation is great, that is for sure

[โ€“] bterwijn@programming.dev 1 points 5 days ago

Thanks a lot, I hope it can bring much value for you.

[โ€“] a_non_monotonic_function@lemmy.world 4 points 6 days ago* (last edited 6 days ago) (1 children)

Like anything else, these sorts if issues are rather murky and are directly impacted by the user's competence in the tools.

Does python have obvious overhead issues? Yes.

Does Python have to be super inefficient? No.

Take basic set operations, for example. Do it manually in the language and it will be dog slow. Do it using the set class? You are leveraging the speed of the underlying C implementation.

E.g., In competitive programming eventually need C or Java, but a strong Python user can move the bar significantly in terms of how many problems are possible with Python.

If you don't want mutability you have to go to a pure functional language like Haskell, but then you have to copy a big list every time you make a change. There are ways to optimize copying by secretly sharing data behind the scene but you pay a performance price in some way.

I don't believe this to be the case. The immutability is precisely why efficient structural sharing is possible without screwing up other data structures. And for standard stuff, it is actually happening behind the scenes already.

You see similar claims about recursion in general, but those claims are often so broad that they don't hold water. I mean, yea, if your language sucks at optimizing recursion it isn't going to be a pleasant experience, but tail call elimination, lazy evaluation, etc. mean you can write some really awesome and efficient code.

I think the bigger problem is that it takes a lot of time to internalize what is going on under the hood when you make a call or initialize a data structure.

[โ€“] bterwijn@programming.dev 3 points 6 days ago (1 children)

I've built a nice visualizer for internalizing what is going on under the hood, I hope that can help people.

[โ€“] a_non_monotonic_function@lemmy.world 2 points 6 days ago (1 children)

Well, that is interesting, isn't it?

Thank you for the lead. I'm setting that link aside. I might use it in class next time.

[โ€“] bterwijn@programming.dev 2 points 5 days ago

Great, I hope it can bring much value for your teaching.

[โ€“] MrWrinkles@leminal.space 11 points 1 week ago (2 children)

As a C programmer, this reads as a shitpost lol. Best I can do is interpret this in Tcl, but I'm lost with |=

[โ€“] a_non_monotonic_function@lemmy.world 2 points 6 days ago* (last edited 6 days ago)

C already has that class of operators, but lacks a set in the base language.

You are probably interacting with analogous operators all day long.

I don't see why extending those notions would cause you issues as a C programmer.

[โ€“] bterwijn@programming.dev 3 points 1 week ago* (last edited 1 week ago) (1 children)

It's mainly targeted to Python programmers at: https://programming.dev/c/python

The Solution link gives you a visualization of program execution, I've started to make the same sort of visualizer for C. What do you think?

[โ€“] MrWrinkles@leminal.space 3 points 1 week ago (1 children)

That's really good. I love the play feature. This motivates me to learn more Python.

[โ€“] bterwijn@programming.dev 2 points 1 week ago

Thanks, I'm pretty happy with it myself.

[โ€“] SpaceNoodle@lemmy.world 4 points 1 week ago (1 children)

First link is broken, second doesn't lead to what's claimed.

[โ€“] bterwijn@programming.dev 2 points 1 week ago* (last edited 1 week ago) (1 children)

Can you tell me what the problem is for you when you click the first link? I tried to make it work for all browsers (even mobile, but the UI isn't very mobile friendly), so I would like to get it working on your system too. If you can give me a bug report (and maybe try from different system), that would be great.

The second link explains:

It's left to the reader to generalize this to x |= y vs x = x | y.

[โ€“] SpaceNoodle@lemmy.world 3 points 1 week ago (1 children)
[โ€“] bterwijn@programming.dev 2 points 1 week ago

Diagnostics came up with some "double percent-encoding" issue on IOS. I've updated the website with a workaround, hopefully it does work now. Thanks for bug reporting.

[โ€“] b34k@lemmy.world 3 points 1 week ago (1 children)

Not too familiar with the union operator for dicts, but that result was not what I expected.

[โ€“] bterwijn@programming.dev 1 points 1 week ago* (last edited 1 week ago)

The dict union operator is similar to that of set, but in addition the right operand overwrites values of keys of the left operand. It's a nice operator once you get used to it.

Easy to make a mistake with this exercise:

  • first realize b = b | {3: []} makes a shallow copy
  • then realize the shallow copy still references the value of key 2 in a so we can still append to that and change a
[โ€“] Daedskin@lemmy.zip 1 points 1 week ago (1 children)

Couldn't get the solution page working on mobile (none of the buttons seemed to do anything); but my guess is it would be {1: []} due to |= silently using __or__ and an assignment, meaning b now holds a new reference

[โ€“] bterwijn@programming.dev 5 points 1 week ago* (last edited 1 week ago)

This is what the solution link is supposed to give you at the end of the execution animation:

It's hard to support all mobile browsers, on some it does work.