placeholder

Synchronization is bad for scale

In the early days of Mailgun I started working on a distributed lock service. Something I had worked on briefly at Rackspace. Even as I implemented the thing, I had the sneaky suspicion that it was a bad idea.

Click to view the original at wippler.dev

Hasnain says:

“Since synchronization is literally built into the database, it’s very tempting to use the Database and the transactions they provide as the synchronization point for EVERYTHING. I would advise caution here, because the database is an expensive part of; and often is THE MOST EXPENSIVE part of your infrastructure. It’s better suited for storing bits on disk, than for synchronizing all of the things for you. If you rely on the DB for synchronization, you will inevitably need to shard and rebalance your database in order to continue scaling, and BTW, you are scaling the most expensive part of your infra.”

Posted on 2024-07-06T22:00:22+0000

placeholder

I Have No Constructor, and I Must Initialize

It has been three days. The room is cold and dark, but your screens are blinding. You feel disoriented as you come in and out of dissociative episodes. Now and again, you laugh, to no accompaniment. Why are you here? Was this your fault?

Click to view the original at consteval.ca

Hasnain says:

“That should be most of it. I mean, there are special initialization rules for static variables (constant initialization included), but, like, do you really care? In my humble opinion, here’s the key takeaway: just write your own fucking constructors! You see all that nonsense? Almost completely avoidable if you had just written your own fucking constructors. Don’t let the compiler figure it out for you. You’re the one in control here. Or is it that you think you’re being cute? You just added six instances of undefined behaviour to your company’s codebase, and now twenty Russian hackers are fighting to pwn your app first. Are you stupid? What’s the matter with you? What were you thinking? God.”

Posted on 2024-07-06T06:23:19+0000

placeholder

Beating NumPy’s matrix multiplication in 150 lines of C code

TL;DR The code from the tutorial is available at matmul.c. This blog post is the result of my attempt to implement high-performance matrix multiplication on CPU while keeping the code simple, portable and scalable. The implementation follows the BLIS design, works for arbitrary matrix sizes, and, wh...

Click to view the original at salykova.github.io

Hasnain says:

“TL;DR The code from the tutorial is available at matmul.c. This blog post is the result of my attempt to implement high-performance matrix multiplication on CPU while keeping the code simple, portable and scalable. The implementation follows the BLIS design, works for arbitrary matrix sizes, and, when fine-tuned for an AMD Ryzen 7700 (8 cores), outperforms NumPy (=OpenBLAS), achieving over 1 TFLOPS of peak performance across a wide range of matrix sizes.”

Posted on 2024-07-05T00:21:31+0000

placeholder

How Uber Migrated Financial Data from DynamoDB to Docstore

Introduction: Each day, Uber moves millions of people around the world and delivers tens of millions of food and grocery orders. This generates a large number of financial transactions that need to be stored with provable completeness, consistency, and compliance. LedgerStore is an immutable, ledger...

Click to view the original at uber.com

Hasnain says:

This is how we measured the success of this critical project:

* Not a single production incident, given how critical money movement is for Uber
* We didn’t have to rollback the migration for a single table
* We backfilled 250 billion unique records and not a single data inconsistency has been detected so far, with the new architecture in production for over 6 months
* Our new data model reduced complexity in the query path, and we observed latency improvements in both read and write path on Docstore
* The project also resulted in technology consolidation, reducing external dependencies

From a business impact perspective, operating LedgerStore is now very cost effective due to reduced spend on DynamoDB. The estimated yearly savings are $6 million per year, and we also laid the foundation for such other future initiatives.

Posted on 2024-07-04T22:45:32+0000

placeholder

Hasnain says:

“If you ever see someone advertise a language feature as “everything is an X”, your first reaction should probably be healthy skepticism, rather than delight. Programming languages generally tend to draw distinctions between distinct things for good reasons.

The C++ committee added concepts after decades of pain with templates, and even these do not overcome all the problems mentioned above. I would not be surprised to see Zig add a similar system (likely with its own twist to it) at some point if it keeps growing in usage.”

Posted on 2024-07-04T02:33:42+0000

placeholder

Gender Quotas and the Crisis of the Mediocre Man

Quotas aren't anathema to meritocracy: they increase competence levels by displacing mediocre men, write Tim Besley, Olle Folke, Torsten Persson and Johanna Rickne. A common criticism against gender quotas is that they are anathema to meritocratic principles. This research on Sweden shows that the o...

Click to view the original at globalpolicyjournal.com

Hasnain says:

“Within each local party, we compare the proportion of competent politicians in elections after the quota to the 1991 level. The figure below show some striking results. The left panel illustrates our estimates for politicians of both genders with black dots showing the change in the proportion of competent representatives in a party which is forced to increase their share of women (by 100 percentage points). The right panel splits the results by men and women (blue dots for men and pink dots for women). It shows distinctly that the average competence of male politicians increased in the places where the quota had a larger impact, and that the effect is concentrated to the three elections following the quota. On average, a higher female representation by 10 percentage points raised the proportion of competent men by 3 percentage points! For the competence of women, we observe little discernible effect.”

Posted on 2024-07-02T15:32:57+0000

placeholder

The 'soft dictatorship' of this Supreme Court

With the nine justices reconvened for the fall term, SFGATE columnist Drew Magary is here to remind us all to stay angry and demand change.

Click to view the original at sfgate.com

Hasnain says:

From October. Prescient.

“The momentum is there, but only if you stay angry. Given this court’s pattern of behavior, that shouldn’t be hard for you to do. Don’t treat these judges with respect, and don’t allow our elected representatives to treat them with respect, either. Tell those reps that this court isn’t legitimate. It’s vile, evil and must be destroyed. Never stop saying it to them.”

Posted on 2024-07-02T02:44:03+0000

placeholder

What does the Supreme Court immunity ruling mean for Trump? 6 questions answered

What happens to Trump’s federal criminal charges, especially if he is reelected? Here are five quick questions about the court’s ruling.

Click to view the original at pbs.org

Hasnain says:

I don’t see how the country comes back from this and the Chevron ruling without massive upheaval and someone willing to take action in the moment (and our current democratic leadership is certainly not cut out for this)

As someone wrote on twitter:

“Thinking about how it would be reported if the ex-president of a non-Western country was ruled immune from prosecution by a panel of judges he himself appointed”

Posted on 2024-07-02T01:41:16+0000

placeholder

A discussion of discussions on AI bias

There've been regular viral stories about ML/AI bias with LLMs and generative AI for the past couple years. One thing I find interesting about discussions of bias is how different the reaction is in the LLM and generative AI case when compared to "classical" bugs in cases where there's a clear bug.....

Click to view the original at danluu.com

Hasnain says:

“We can observe something similar in almost every consumer market and many B2B markets as well, and that's when we're talking about issues that have known solutions. If we look at problem that, from a technical standpoint, we don't know how to solve well, like subtle or even not-so-subtle bias in ML models, it stands to reason that we should expect to see more and worse bugs than we'd expect out of "classical" software systems, which is what we're seeing. Any solution to this problem that's going to hold up in the market is going to have to be robust against the issue that consumers will overwhelmingly choose the buggier product if it has more features they want or ships features they want sooner, which puts any solution that requires taking care in a way that significantly slows down shipping in a very difficult position, absent a single dominant player, like Intel in its heyday.”

Posted on 2024-07-01T02:24:58+0000

placeholder

IPC in Rust - a Ping Pong Comparison

Home »  Posts »  IPC in Rust - a Ping Pong Comparison IPC in Rust - a Ping Pong Comparison 2024-06-12 · 20 min ·   | Suggest Changes Table of Contents The Problem A Note on Timing Benchmarking Approach 1 - Pipes Approach 2 - TCP Approach 3 - UDP Approach 4 - Shared Memory Results Syste...

Click to view the original at 3tilley.github.io

Hasnain says:

“If I had to do this in Production, for the majority of workloads I'd probably still use an HTTP / TCP connection. It's portable, reliable on message failure, and I could split it across machines if needs be. However for the cases where latency really matters, the maintenance overhead of using shared memory is worth it.”

Posted on 2024-07-01T02:03:05+0000