Tales of the M1 GPU - Asahi Linux
marcan asked me to write an article about the M1 GPU, so here we are~! It’s been a long road over the past few months and there’s a lot to cover, so I hope you enjoy it!
Really cool technical story of how they built a kernel + user space driver for the M1 GPU so it could run Linux. And some great hackery in between. I promise I didn’t share this just for the Rust fanboyism.
“Normally, when you write a brand new kernel driver as complicated as this one, trying to go from simple demo apps to a full desktop with multiple apps using the GPU concurrently ends up triggering all sorts of race conditions, memory leaks, use-after-free issues, and all kinds of badness.
But all that just… didn’t happen! I only had to fix a few logic bugs and one issue in the core of the memory management code, and then everything else just worked stably! Rust is truly magical! Its safety features mean that the design of the driver is guaranteed to be thread-safe and memory-safe as long as there are no issues in the few unsafe sections. It really guides you towards not just safe but good design.”Posted on 2022-11-30T04:30:36+0000
Pointer compression in Oilpan · V8
Pointer compression in Oilpan allows for compressing C++ pointers and reducing the heap size by up to 33%.
This continues to increase my motivation to learn more about garbage collectors.
“The numbers reported represent the 50th and 99th percentile for Blink memory allocated with Oilpan across the fleet. The reported data shows the delta between Chrome 105 and 106 stable versions. The absolute numbers in MB give an indication on the lower bound that users can expect to see. The real improvements are generally a bit higher due to indirect effects on Chrome’s overall memory consumption. The larger relative improvement suggests that packing of data is better in such cases which is an indicator that more memory is used in collections (e.g. vectors) that have good packing. The improved padding of structures landed in Chrome 108 and showed another 4% improvement on Blink memory on average.
Because Oilpan is ubiquitous in Blink, the performance cost can be estimated on Speedometer2. The initial prototype based on a thread-local version showed a regression of 15%. With all the aforementioned optimizations we did not observe a notable regression.”Posted on 2022-11-29T16:53:26+0000
Goodbye, Data Science
This is more of a personal post than something intended to be profound. If you are looking for a point, you will not find one here. Frankly I am not even sure who the target audience is for this (p…
“But there’s also a part of me that’s just like, how can you not be curious? How can you write Python for 5 years of your life and never look at a bit of source code and try to understand how it works, why it was designed a certain way, and why a particular file in the repo is there? How can you fit a dozen regressions and not try to understand where those coefficients come from and the linear algebra behind it? I dunno, man.
Ultimately nobody really knows what they are doing, and that’s OK. But between companies not building around this observation, and individuals not self-directing their educations around this observation, it is just a bit maddening to feel stuck in stupid hell.”Posted on 2022-11-29T16:44:25+0000
Retrofitting null-safety onto Java at Meta
Nullsafe is a new static analysis tool that is used at Meta to detect NullPointerException (NPE) errors in Java code.
Great read. Beyond the details of the static analysis itself, I enjoyed the insights into how they rolled out this program at scale and the lessons learned throughout.
"The problems outlined above are hardly specific to Meta. Unexpected null-dereferences have caused countless problems in different companies. Languages like C# evolved into having explicit nullness in their type system, while others, like Kotlin, had it from the very beginning.
When it comes to Java, there were multiple attempts to add nullness, starting with JSR-305, but none was widely successful. Currently, there are many great static analysis tools for Java that can check nullness, including CheckerFramework, SpotBugs, ErrorProne, and NullAway, to name a few. In particular, Uber walked the same path by making their Android codebase null-safe using NullAway checker. But in the end, all the checkers perform nullness analysis in different and subtly incompatible ways. The lack of standard annotations with precise semantics has constrained the use of static analysis for Java throughout the industry."Posted on 2022-11-29T06:07:41+0000
Please Restore Our Registers When You’re Done With Them
“Hey, you. Yes you, that function over there. When you’re cleaning up please remember to restore all of my registers. Yes, that one too – what do you think this is, Linux?”That’s the problem I was …
Assembly is so hard to get right. More so in a cross-platform manner.
"I decided to write this up because I thought that the journey was interesting, but also because the journey is not over. There could be other registers that are not being properly saved and restored in Chromium. There could be other projects that are making this mistake, perhaps not aware of the differences between the Linux and Windows ABI. Any software rules that are not tested and enforced are inevitably broken and I am not aware of any consistent testing to detect ABI violations. More bugs of this type seem inevitable."Posted on 2022-11-29T06:01:04+0000
Coping strategies for the serial project hoarder
I gave a talk at DjangoCon US 2022 in San Diego last month about productivity on personal projects, titled “Massively increase your productivity on personal projects with comprehensive documentation and …
Bookmarking this for later rereading. some of this is stuff that's worked well for me, and a lot of it was novel stuff. I'm a huge fan of issue driven development and documenting via commits, for what it's worth.
The quote though, uhhhhh....
"This is the most important tip: avoid side projects with user accounts.
If you build something that people can sign into, that’s not a side-project, it’s an unpaid job. It’s a very big responsibility, avoid at all costs!
Almost all of my projects right now are open source things that people can run on their own machines, because that’s about as far away from user accounts as I can get.
I still have a responsibility for shipping security updates and things like that, but at least I’m not holding onto other people’s data for them."Posted on 2022-11-28T06:09:50+0000
Index Merges vs Composite Indexes in Postgres and MySQL
Index Merges vs Composite Indexes in Postgres and MySQLNov 2022Composite indexes are about 10x faster than index merges. In Postgres, the gap is larger than in MySQL because Postgres doesn't support index-only scans for queries that involve index merges.Napkin MathComposite Index: ~1msIndex Merge: ~...
“If you’re wondering if you need to add a composite index, or can get away with creating to single indexes and rely on the database to use both indexes — then the rule of thumb we establish is that an index merge will be ~10x slower than the composite index. However, we’re still talking less than 100ms in most cases, as long as you’re operating on 100s of rows (which in a relational, operational database, hopefully you mostly are).”Posted on 2022-11-28T00:53:15+0000
Recognizing patterns in memory
Something I find frustrating is how hard it is to teach debugging skills. I think the biggest reason is because there are many things that can only be learned through experience. This is true for anything that requires pattern recognition. Our brains are great at recognizing patterns, but it often t...
Parents of Ex-Stanford Goalie Suing School for Wrongful Death
Katie Meyer’s family claims Stanford’s handling of a disciplinary process involving their late daughter led to her death.
““Katie Meyer’s tragic death resulted from Stanford’s egregious and reckless mishandling of its disciplinary process,” attorney Kim Dougherty said in a statement. “Stanford has known for years that its disciplinary process, in its own Committee 10’s words, is ‘overly punitive’ and harmful to its students, yet the school and its administrators have done nothing to correct its procedures. Through this litigation we will not only obtain justice for Katie, but also ensure necessary change is put into place to help protect Stanford students and provide safeguards when students are in need of support.””Posted on 2022-11-25T18:01:58+0000
Twitter king Dril on Musk’s chaotic reign
The ‘patron saint of the internet’ tells The Post he’ll never pay for verification but will learn to code if Musk offers him a job.
When twitter dies, I’ll pour one out for Dril.
“So far, Dril said, he’s enjoying the spectacle of Musk’s takeover. “Elon seems like one of the classic comedic showmen,” he said. “Everything he does is a comedic bit. He’s always trying to get a laugh, that’s why he makes all his cars suicidal. Just watching everything burn, it’s entertaining, that’s for sure.””Posted on 2022-11-24T05:48:13+0000