What is the Python GIL, how it works, and how it affects gunicorn.

Photo by John Anvik on Unsplash

What Gunicorn worker type should I choose for production?

This is a recurring question I get asked and something I tried to explain in a previous article.

After discussing the topic with other people I understood that the article was a bit incomplete without exposing the internals of Python that directly influence how Gunicorn serves requests.

As we saw…


How to choose between different analyzers and queries to get the best search performance? Benchmarking of course!

Photo by Arie Wubben on Unsplash

Deploying a large-scale full-text search engine can be very hard. Elasticsearch makes the job much easier but it’s not one size fits all — quite the contrary.

Elasticsearch has many configurations and features…


How some simple changes can result in less latency and better memory usage.

Redis Strings are probably the most used (and abused) Redis data structure.

One of their main advantages is that they are binary-safe — This means you can save any type of binary data in Redis.

But as…


In this post, I’ll describe the building blocks of a resilient self-hosted transcoding platform using open source tools and AWS.

For part two, I’ll share a sample python project that allows you to bootstrap this in minutes.

General principles

When building a system like this, you should never compromise on these:

  • Self-healing…


Building a plugin to filter large lists of numbers and get 10x performance on Elasticsearch cluster.

A few years ago, I faced a bottleneck in ElasticSearch when trying to filter on a big list of integer ids. …


Increase your python code performance and security without changing the project source code.

Table of Contents

  • Introduction
  • Motivation
  • Benchmarks
  • Further Optimizations
  • The Perfect Dockerfile for Python
Photo by SpaceX on Unsplash

Introduction

Having a reliable Dockerfile as your base can save you hours of headaches and bigger problems down the road.

This post will share the “perfect” Python Dockerfile. Of course, there is no such thing as perfection and I’ll gladly accept…


Active-Active multi-region is challenging and expensive, but sometimes it’s the only choice.

In my previous article (you can read it here), I showed the architecture used to handle a large-scale sneakers drop backend.

There was an essential part missing though, especially in our case with the strong requirement of “first come, first served”.

If the machines are in the USA and you’re…


A constant flow of document updates can bring an Elasticsearch cluster to its knees. Fortunately, there are ways to avoid that scenario.

As we’ve seen in my previous article, Elasticseach doesn’t really support updates. In Elasticsearch, an update always means delete+create.

In a previous project, we were using Elasticsearch for full-text search and needed to save some signals, like new followers, along with the user document.

That represented a big issue since…


Multiple strategies that you can use to increase Elasticsearch write capacity for batch jobs and/or online transactions.

Over the last few years, I’ve faced bottlenecks and made many mistakes with different ES clusters when it comes to its write capacity. Especially when one of the requirements is to write into a live Index that has strict SLAs for reading operations.

If you use Elasticsearch in production environments…


How to build a backend that can handle millions of concurrent users efficiently and consistently.

Photo by Hermes Rivera on Unsplash

Context

Brands like Nike, Adidas, or Supreme created a new trend in the market called “drops”, where they release a finite amount of items. …

Luis Sena

Principal Engineer @ Farfetch https://www.linkedin.com/in/luis-sena/

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store