* . *
  • About
  • Advertise
  • Privacy & Policy
  • Contact
Monday, October 6, 2025
Earth-News
  • Home
  • Business
  • Entertainment
    Row K Entertainment Emerges as Major New Hollywood Buyer With Splashy TIFF Shopping Spree – TheWrap

    Row K Entertainment Emerges as Major New Hollywood Buyer With Splashy TIFF Shopping Spree – TheWrap

    Charlie Hunnam Reflects on Playing a Serial Killer in Monster: The Ed Gein Story – Yahoo

    Charlie Hunnam Reveals the Dark Challenges of Playing a Serial Killer in Monster: The Ed Gein Story

    “Reba” cast, then and now: See the stars 24 years later (and who’s reunited for another show) – Yahoo

    “Reba” cast, then and now: See the stars 24 years later (and who’s reunited for another show) – Yahoo

    Why Taylor Swift Name-Dropped Elizabeth Taylor in Her New Album – Yahoo

    Here’s Why Taylor Swift Dropped Elizabeth Taylor’s Name in Her New Album

    Al Roker Gives Olivia Dean an Unexpected ‘New Job’ on the ‘Today’ Show – Yahoo

    Al Roker Shocks Olivia Dean with an Exciting New Role on the ‘Today’ Show

    Books about the arts and some haunts for a Denton October – Denton Record-Chronicle

    Uncover Artistic Treasures and Spooky Adventures to Experience in Denton This October

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Forget Cowbells. Cows Wear High-Tech Collars Now. – The New York Times

    Ditch the Cowbells: Discover the High-Tech Collars Transforming Cattle Care

    What the Recent Price Surge Means for Figure Technology Solutions After SEC Settlement – Yahoo Finance

    What the Recent Price Surge Reveals About Figure Technology Solutions Following SEC Settlement

    MAC Brings iPad Technology to Football Sidelines Across All 13 Member Schools – Sports Video Group

    MAC Brings iPad Technology to Football Sidelines Across All 13 Member Schools – Sports Video Group

    Technology Is Becoming More Important Than Humans In CX – No Jitter

    Technology Is Becoming More Important Than Humans In CX – No Jitter

    A Tech Expo Shows What China Can Make, but Not Who’ll Buy It All – The New York Times

    Inside China’s Tech Expo: Cutting-Edge Innovations Face Uncertain Demand

    Steampunk Metal Oval Technology Sense Sunglasses Personality Handmade Chain Multicolor Sunglasses UV400 – The San Joaquin Valley Sun

    Steampunk Metal Oval Sunglasses with Handmade Multicolor Chain – Bold UV400 Protection and Unique Style

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
  • Home
  • Business
  • Entertainment
    Row K Entertainment Emerges as Major New Hollywood Buyer With Splashy TIFF Shopping Spree – TheWrap

    Row K Entertainment Emerges as Major New Hollywood Buyer With Splashy TIFF Shopping Spree – TheWrap

    Charlie Hunnam Reflects on Playing a Serial Killer in Monster: The Ed Gein Story – Yahoo

    Charlie Hunnam Reveals the Dark Challenges of Playing a Serial Killer in Monster: The Ed Gein Story

    “Reba” cast, then and now: See the stars 24 years later (and who’s reunited for another show) – Yahoo

    “Reba” cast, then and now: See the stars 24 years later (and who’s reunited for another show) – Yahoo

    Why Taylor Swift Name-Dropped Elizabeth Taylor in Her New Album – Yahoo

    Here’s Why Taylor Swift Dropped Elizabeth Taylor’s Name in Her New Album

    Al Roker Gives Olivia Dean an Unexpected ‘New Job’ on the ‘Today’ Show – Yahoo

    Al Roker Shocks Olivia Dean with an Exciting New Role on the ‘Today’ Show

    Books about the arts and some haunts for a Denton October – Denton Record-Chronicle

    Uncover Artistic Treasures and Spooky Adventures to Experience in Denton This October

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Forget Cowbells. Cows Wear High-Tech Collars Now. – The New York Times

    Ditch the Cowbells: Discover the High-Tech Collars Transforming Cattle Care

    What the Recent Price Surge Means for Figure Technology Solutions After SEC Settlement – Yahoo Finance

    What the Recent Price Surge Reveals About Figure Technology Solutions Following SEC Settlement

    MAC Brings iPad Technology to Football Sidelines Across All 13 Member Schools – Sports Video Group

    MAC Brings iPad Technology to Football Sidelines Across All 13 Member Schools – Sports Video Group

    Technology Is Becoming More Important Than Humans In CX – No Jitter

    Technology Is Becoming More Important Than Humans In CX – No Jitter

    A Tech Expo Shows What China Can Make, but Not Who’ll Buy It All – The New York Times

    Inside China’s Tech Expo: Cutting-Edge Innovations Face Uncertain Demand

    Steampunk Metal Oval Technology Sense Sunglasses Personality Handmade Chain Multicolor Sunglasses UV400 – The San Joaquin Valley Sun

    Steampunk Metal Oval Sunglasses with Handmade Multicolor Chain – Bold UV400 Protection and Unique Style

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
Earth-News
No Result
View All Result
Home General

Scaling Argo Repo Server: How Increasing Replicas Solved Startup Delays

July 2, 2024
in General
Scaling Argo Repo Server: How Increasing Replicas Solved Startup Delays
Share on FacebookShare on Twitter

Man pointing at screen. Screen is surrounded by images related to software engineering.

“Engineering in the Web” (Source: publicdomainpictures.net) 

Handling resource constraints is never easy. Technical issues, confusing configurations, and cost considerations make engineering the best use of resources challenging. 

We recently had one of our continuous deployment components, the Argo repo server, start to hit resource constraints. We will dive deeper into the component later in the article, but the repo server’s function is to fetch content from a GitHub repository. Having the necessary files and manifests is crucial for deploying applications, and a slow repo server can significantly degrade deployment speeds—and that’s exactly what we just experienced. Applications at Realtor.com took around 10 minutes from creation to resource deployment, but that time increased to over 20 minutes on average and hours in the worst cases. By improving the repo server and solving the technical issues to increase the number of replicas, we reduced deployment times from over 20 minutes to an average of just 5 minutes—a 300% improvement!

In this article, we’ll walk through our setup, our challenges, and how we arrived at the final solution.

What is Continuous Deployment?

To better understand the situation, let’s briefly review some DevOps principles, namely CI/CD.

CI/CD stands for continuous integration and continuous deployment. Continuous integration is the software engineering practice of making changes frequently while automatically testing, building, and monitoring these changes. Instead of manually performing all the steps necessary to get a release ready, automated steps are performed for every change. Continuous deployment is the practice of automatically creating resources without manually performing releases. Instead of manually performing releases and waiting on specific cycles, resources are created automatically on every change.

For example, a change pushed to GitHub is automatically tested, packaged, deployed, and monitored with minimal to no manual intervention. This process allows engineers to focus more on application development rather than spending hours planning and manually handling releases.

ArgoCD

For continuous deployment, we use a tool called ArgoCD. This delivery tool integrates with GitHub and deploys applications to Kubernetes. Argo fetches configurations from GitHub and deploys those applications into Kubernetes. Using Argo ensures that all our configurations are declarative and managed as code.

Repo Server

The Argo repo server interacts with GitHub to retrieve the necessary contents from a repository. It can list repositories, folders, and files in GitHub to find what it needs, and it can fetch (download) the files required to deploy the desired application.

Both listing and fetching to Github require making API calls over the internet. When there are many applications, the repo server makes numerous network calls. Too many calls can cause a bottleneck in the repo server’s performance, leading to delays and disruptions like those mentioned earlier. Why not just scale the repo server? Increasing the number of pods in Kubernetes is simple, right? Scaling the repo server is indeed the right answer, but there was a hurdle we had to overcome first.

The Problem

To handle numerous network calls, the repo server employs caching. Caching saves information closer to the user so that when needed again, it is easier and faster to retrieve. For the repo server, this means caching the response of GitHub calls so that when it needs to make the same call, it can pull from the cache instead of making a full network call.

The result of this caching needs to be stored somewhere. We didn’t want to store it locally on the machine running Argo, as that wouldn’t scale well over time. At Realtor.com, we use AWS, so our choice for file storage was Amazon Elastic File Storage (EFS). The file storage is mounted onto the repo server, and the folders and files used for caching are stored in EFS, just like any other files.

Caching and EFS are great, but they also introduced issues. The folder mounted onto the repo server for EFS held the information needed to make and maintain connections to GitHub. That information is specific to each instance of the repo server. When another replica was added, the mount used was the exact same path in EFS. This meant the information needed to make GitHub connections and the cache would be overwritten constantly by the other replicas, causing failures in cache usage and GitHub connections. EFS and shared files were great in theory but disastrous in practice.

The Solution

The requirements for the solution were clear: continue using caching and EFS. Caching was necessary because network bottlenecks would persist without it, even with scaled repo server replicas. EFS was necessary because it was easy to use and efficiently scaled the files used by caching.

The solution was not to have every repo server replica mount in the same folder in EFS. This is simple enough, but how do we do that in Kubernetes to make each mount unique? 

As of Kubernetes version 1.17, you can use supPathExpr, which allows you to specify an expression that will be used as the path to be created in your volume (in our case, EFS). Without this, the usual code is mountPath: /example/path, but with it, the expression looks like this: subPathExpr: $(POD_NAME). This expression ensures the folder created in EFS by each pod will have the same name as the pod, making every folder in EFS unique. Each replica of the repo server now gets its own folder in EFS – no more collisions!

Having individual EFS folders and no more collisions allowed the repo server to have multiple replicas. The small increase from one replica to three enabled more connections to happen simultaneously, drastically reducing deployment times in Argo.

Since Argo is already set up to use multiple instances of the repo server, increasing the number of replicas was the only additional configuration needed. This means that as more projects are added, if latency recurs, we can increase the number of replicas again to scale along with our growth and Kubernetes adoption.

To summarize, adding just one line of code resulted in a 300% improvement in deployment time. While it was an easy change, it wasn’t simple. In the future, this opens up possibilities of adding pod autoscaling (HPA), which will automatically adjust the number of replicas based on demand, further enhancing our ability to automate processes and respond quickly to engineers’ needs.

It’s amazing to see how much thought and understanding are needed for such a simple solution, highlighting the complexity and beauty of engineering.

Profile of Nathan Moore

Nathan Moore is a DevOps engineer at Realtor.com with 3 years of experience in software engineering. Nathan is also an AWS Certified Solutions Architect – Associate.

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : Realtor.com – https://techblog.realtor.com/scaling-argo-repo-server-how-increasing-replicas-solved-startup-delays

Previous Post

Israeli launches airstrikes on areas east of Khan Younis after Palestinian Islamic Jihad rocket attacks

Next Post

Why Defining Values Is Critical To Shaping How Kids Use AI

World Mental Health Day Festival and gala set for this week in NYC – MSNBC News

World Mental Health Day Festival and gala set for this week in NYC – MSNBC News

October 6, 2025
Green economy means jobs for NJ, report says – NJ Spotlight News

Green economy means jobs for NJ, report says – NJ Spotlight News

October 6, 2025
Row K Entertainment Emerges as Major New Hollywood Buyer With Splashy TIFF Shopping Spree – TheWrap

Row K Entertainment Emerges as Major New Hollywood Buyer With Splashy TIFF Shopping Spree – TheWrap

October 6, 2025
Republicans are holding Americans’ health care hostage — and lying about it – MSNBC News

How Republicans Are Holding Americans’ Health Care Hostage – and Deceiving the Public

October 6, 2025
Irish language: Naomi Long says politicians are playing out ‘culture war’ – BBC

Naomi Long Slams Politicians for Igniting a Divisive ‘Culture War’ Over the Irish Language

October 6, 2025
Air quality speed run – Washington State Department of Ecology (.gov)

Battling Pollution: Washington State’s Urgent Fight for Cleaner Air

October 6, 2025
Equity in science is a beautiful lie — and I’m done pretending – Nature

Equity in science is a beautiful lie — and I’m done pretending – Nature

October 6, 2025
Explore a bird database with 11,500 species – Popular Science

Explore an Incredible Bird Database Showcasing 11,500 Stunning Species

October 6, 2025
South Shore broker funded ‘extravagant lifestyle’ with $11.6 million taken from clients and colleagues, prosecutors say – Boston.com

South Shore broker funded ‘extravagant lifestyle’ with $11.6 million taken from clients and colleagues, prosecutors say – Boston.com

October 6, 2025
QRG Capital Management Inc. Purchases 9,519 Shares of Marvell Technology, Inc. $MRVL – MarketBeat

QRG Capital Management Takes Bold Step by Acquiring 9,519 Shares of Marvell Technology

October 6, 2025

Categories

Archives

October 2025
M T W T F S S
 12345
6789101112
13141516171819
20212223242526
2728293031  
« Sep    
Earth-News.info

The Earth News is an independent English-language daily published Website from all around the World News

Browse by Category

  • Business (20,132)
  • Ecology (854)
  • Economy (875)
  • Entertainment (21,749)
  • General (17,446)
  • Health (9,917)
  • Lifestyle (887)
  • News (22,149)
  • People (876)
  • Politics (886)
  • Science (16,085)
  • Sports (21,375)
  • Technology (15,855)
  • World (858)

Recent News

World Mental Health Day Festival and gala set for this week in NYC – MSNBC News

World Mental Health Day Festival and gala set for this week in NYC – MSNBC News

October 6, 2025
Green economy means jobs for NJ, report says – NJ Spotlight News

Green economy means jobs for NJ, report says – NJ Spotlight News

October 6, 2025
  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

Go to mobile version