🌎️ Scaling to billions

Hey everybody.

Today we’ll take a look at how India’s biggest streaming site scaled its reaction system into the billions. Plus, how one guy hacked the points system behind all major airlines and why CSV is still king.

Scalability

Scaling India’s Biggest Streaming Site
Have you ever wondered how live emojis and reactions appear in real-time during a live stream? Hotstar, India's largest streaming platform, built this feature, and Dedeepya who was a Full Stack engineer detailed the entire process in a blog post.

Key Principles:

Scalability: The system needed to handle growing traffic, so horizontal scalability was crucial. By using load balancers and auto-scaling configurations, the system dynamically adjusts resources based on demand.

Decomposition: Breaking down the system into smaller, independent components allows each to scale as needed. This modularity ensures that each part performs its task efficiently without being a bottleneck.

Asynchronous Processing: To support high concurrency without blocking resources, asynchronous processing was essential. This approach also reduces the load on the client by processing emojis offline.

How It Works:

When viewers send emojis, the requests are handled via an HTTP API. To prevent delays, these requests are written to a local buffer, then asynchronously processed and sent to a message queue like Kafka. For real-time processing, Hotstar chose Spark for its micro-batching and aggregation capabilities, essential for delivering a real-time experience.

Data Delivery: Finally, messages are delivered through Hotstar's real-time messaging infrastructure, PubSub, which has a separate write-up.

Hotstar has captured over 6.5 billion emojis with this architecture, ensuring a seamless experience for millions of users.

For a deeper dive, check out the full blog post.

Quick Links

➗ Why CSV is still king
CSV wasn’t a planned invention—it just kind of happened. In the chaotic world of data, CSV is like the cockroach of file formats: unassuming, indestructible, and everywhere. While fancier formats have flashed by, CSV has silently dominated the data game. So, what’s the secret behind its quiet, unstoppable rise?

💳️ It took 2 years to fix Stripe’s Payments API
Stripe took two years to rewrite their payments API after realizing they’d made a huge mistake. With deep pockets, they could afford to fix it. But for the rest of us, a rewrite is a luxury we can’t afford—we’ve got to nail it on the first try. So what did Stripe do wrong?

📝 7 Things I've learned building a modern TUI Framework
Will McGugan has spent over a year working on Textual, and along the way, he’s uncovered some interesting insights about terminals in Python and software development as a whole. Here’s a glimpse into what he’s learned.

🛩️ Hacking the Largest Airline and Hotel Rewards Platform
Sam Curry discovered multiple security flaws in points.com, the backend for many airline and hotel rewards programs. These vulnerabilities could have let attackers access sensitive customer data. Also, attackers could transfer points from accounts or gain access to a global admin site, allowing them to control rewards programs and manage customer accounts.

With the issues now patched, Sam shares an overview of the vulnerabilities, along with detailed technical write-ups for each one.

Development

How to grow as a software engineer

Ruslan’s growth as a software engineer has been shaped by a few key principles, learned early in his career, that continue to guide him even now. I think that the final point is very important.

Drive
Ambition isn't something you can learn—it's an inner will that either pushes you forward or leaves you standing still. For Ruslan, that drive was a constant companion, propelling him to dive into new technologies like Python and CI/CD, even teaching them while he was still learning himself.

Delivering Results
Technical skills are important, but consistently delivering results is what truly sets you apart. Ruslan put in the extra effort, often leaning into workaholic tendencies, especially on projects that excited him. This commitment helped build his reputation and credibility, essential for long-term success.

Choosing the Right Projects
Working on high-impact projects that align with personal interests can be a game-changer. These projects not only provide visibility and future opportunities but also keep you motivated when challenges arise—and they inevitably will.

Craft
Delivering with both speed and quality requires deep expertise. Ruslan found that expanding his knowledge broadly while diving deep into specific technologies and achieving T-shaped expertise was key to making a meaningful impact.

Teaching
Teaching others is one of the best ways to solidify your understanding and identify gaps in your knowledge. It also positions you as a go-to person in your field. For Ruslan, even well-executed code reviews became a form of teaching, helping both himself and others grow.

Tolerance for Conflict
Criticism is part of the journey, whether it’s constructive feedback or just plain negativity. Ruslan learned this the hard way, through missed deadlines and less-than-perfect feedback. His advice? Develop a thick skin early on, but be strategic—know when to stand your ground and when to let things slide.

Learning How Business Works
This lesson took Ruslan a long time to grasp, but it's one of the most important. Understanding how the business operates helps you identify high-impact projects and positions you as a valued partner, not just another tech person in the background.

These principles have not only shaped Ruslan's early career but continue to influence his path today and I think some of these are great to keep in mind.

It's funny, people often ask me, What do you even do?

Are you a developer? Just a YouTuber? How do you make a living?

Well, I love software development. Since the day I wrote my first code, it was something I HAD to do.

In learning to code and becoming a programmer I also had the desire to document my journey, my learning, my job changes, creating tutorials, whatever, it was a parallel interest of mine to not only code but create content.

Here's the story.

Last Week’s Result (It seems everybody is using AI!)

How often do you use AI tools in life and work?

🟩🟩🟩🟩🟩🟩 🧑‍💻 Everyday!
🟩🟩🟩🟩🟩🟩 🤷 Occasionally
🟩⬜️⬜️⬜️⬜️⬜️ ❌ Never

Until next week,

Travis.