* . *
  • About
  • Advertise
  • Privacy & Policy
  • Contact
Wednesday, July 23, 2025
Earth-News
  • Home
  • Business
  • Entertainment
    AP Entertainment SummaryBrief at 1:51 p.m. EDT – Channel 3000

    Entertainment Highlights: Key Updates You Can’t Miss

    ‘Devil Wears Prada 2’ casts Anne Hathaway’s love interest replacing Adrian Grenier’s Nate – Entertainment Weekly

    Devil Wears Prada 2′ Casts New Love Interest for Anne Hathaway, Replacing Adrian Grenier’s Nate

    12 ‘Late Show’ Moments Proving Stephen Colbert Can’t Be Replaced – The Mountaineer

    12 Unforgettable ‘Late Show’ Moments That Prove Stephen Colbert Is Truly One of a Kind

    Canes owner Tom Dundon’s real estate firm eyes entertainment complex near RDU – The Business Journals

    Canes Owner Tom Dundon’s Real Estate Firm Unveils Plans for Thrilling New Entertainment Complex Near RDU

    Inspired Entertainment, Inc.’s (NASDAQ:INSE) Price Is Right But Growth Is Lacking After Shares Rocket 29% – simplywall.st

    Inspired Entertainment Soars 29% but Growth Momentum Falls Short

    Kroger shares summer entertainment tips – Supermarket Perimeter

    Ultimate Summer Entertainment Tips to Make Your Season Unforgettable

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Morgan Adamski Joins PwC in Newly Created Cyber, Data & Technology Risk Division – HSToday

    Morgan Adamski Leads the Charge in PwC’s Cutting-Edge Cyber, Data & Technology Risk Division

    Transformative technology, support for food entrepreneurs spotlighted – University of Hawaii System

    How Cutting-Edge Technology and Strong Support Are Revolutionizing Food Entrepreneurs’ Success

    Cold Weather Alloy Opens New Possibilities for Space Technology – Universe Today

    Revolutionary Cold Weather Alloy Unlocks New Frontiers in Space Technology

    Defence Holdings PLC Transitions from Esports to Pioneering Defence Technology

    Bull of the Day: Credo Technology Group (CRDO) – Yahoo Finance

    Bull of the Day: Why Credo Technology Group (CRDO) Is Poised for a Breakout

    BlackSky Technology Inc. Stock Analysis and Forecast – Explosive wealth accumulation – Jammu Links News

    BlackSky Technology Inc.: Unlocking Explosive Wealth Potential Through Expert Stock Analysis and Forecast

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
  • Home
  • Business
  • Entertainment
    AP Entertainment SummaryBrief at 1:51 p.m. EDT – Channel 3000

    Entertainment Highlights: Key Updates You Can’t Miss

    ‘Devil Wears Prada 2’ casts Anne Hathaway’s love interest replacing Adrian Grenier’s Nate – Entertainment Weekly

    Devil Wears Prada 2′ Casts New Love Interest for Anne Hathaway, Replacing Adrian Grenier’s Nate

    12 ‘Late Show’ Moments Proving Stephen Colbert Can’t Be Replaced – The Mountaineer

    12 Unforgettable ‘Late Show’ Moments That Prove Stephen Colbert Is Truly One of a Kind

    Canes owner Tom Dundon’s real estate firm eyes entertainment complex near RDU – The Business Journals

    Canes Owner Tom Dundon’s Real Estate Firm Unveils Plans for Thrilling New Entertainment Complex Near RDU

    Inspired Entertainment, Inc.’s (NASDAQ:INSE) Price Is Right But Growth Is Lacking After Shares Rocket 29% – simplywall.st

    Inspired Entertainment Soars 29% but Growth Momentum Falls Short

    Kroger shares summer entertainment tips – Supermarket Perimeter

    Ultimate Summer Entertainment Tips to Make Your Season Unforgettable

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Morgan Adamski Joins PwC in Newly Created Cyber, Data & Technology Risk Division – HSToday

    Morgan Adamski Leads the Charge in PwC’s Cutting-Edge Cyber, Data & Technology Risk Division

    Transformative technology, support for food entrepreneurs spotlighted – University of Hawaii System

    How Cutting-Edge Technology and Strong Support Are Revolutionizing Food Entrepreneurs’ Success

    Cold Weather Alloy Opens New Possibilities for Space Technology – Universe Today

    Revolutionary Cold Weather Alloy Unlocks New Frontiers in Space Technology

    Defence Holdings PLC Transitions from Esports to Pioneering Defence Technology

    Bull of the Day: Credo Technology Group (CRDO) – Yahoo Finance

    Bull of the Day: Why Credo Technology Group (CRDO) Is Poised for a Breakout

    BlackSky Technology Inc. Stock Analysis and Forecast – Explosive wealth accumulation – Jammu Links News

    BlackSky Technology Inc.: Unlocking Explosive Wealth Potential Through Expert Stock Analysis and Forecast

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
Earth-News
No Result
View All Result
Home General

Scaling Argo Repo Server: How Increasing Replicas Solved Startup Delays

July 2, 2024
in General
Scaling Argo Repo Server: How Increasing Replicas Solved Startup Delays
Share on FacebookShare on Twitter

Man pointing at screen. Screen is surrounded by images related to software engineering.

“Engineering in the Web” (Source: publicdomainpictures.net) 

Handling resource constraints is never easy. Technical issues, confusing configurations, and cost considerations make engineering the best use of resources challenging. 

We recently had one of our continuous deployment components, the Argo repo server, start to hit resource constraints. We will dive deeper into the component later in the article, but the repo server’s function is to fetch content from a GitHub repository. Having the necessary files and manifests is crucial for deploying applications, and a slow repo server can significantly degrade deployment speeds—and that’s exactly what we just experienced. Applications at Realtor.com took around 10 minutes from creation to resource deployment, but that time increased to over 20 minutes on average and hours in the worst cases. By improving the repo server and solving the technical issues to increase the number of replicas, we reduced deployment times from over 20 minutes to an average of just 5 minutes—a 300% improvement!

In this article, we’ll walk through our setup, our challenges, and how we arrived at the final solution.

What is Continuous Deployment?

To better understand the situation, let’s briefly review some DevOps principles, namely CI/CD.

CI/CD stands for continuous integration and continuous deployment. Continuous integration is the software engineering practice of making changes frequently while automatically testing, building, and monitoring these changes. Instead of manually performing all the steps necessary to get a release ready, automated steps are performed for every change. Continuous deployment is the practice of automatically creating resources without manually performing releases. Instead of manually performing releases and waiting on specific cycles, resources are created automatically on every change.

For example, a change pushed to GitHub is automatically tested, packaged, deployed, and monitored with minimal to no manual intervention. This process allows engineers to focus more on application development rather than spending hours planning and manually handling releases.

ArgoCD

For continuous deployment, we use a tool called ArgoCD. This delivery tool integrates with GitHub and deploys applications to Kubernetes. Argo fetches configurations from GitHub and deploys those applications into Kubernetes. Using Argo ensures that all our configurations are declarative and managed as code.

Repo Server

The Argo repo server interacts with GitHub to retrieve the necessary contents from a repository. It can list repositories, folders, and files in GitHub to find what it needs, and it can fetch (download) the files required to deploy the desired application.

Both listing and fetching to Github require making API calls over the internet. When there are many applications, the repo server makes numerous network calls. Too many calls can cause a bottleneck in the repo server’s performance, leading to delays and disruptions like those mentioned earlier. Why not just scale the repo server? Increasing the number of pods in Kubernetes is simple, right? Scaling the repo server is indeed the right answer, but there was a hurdle we had to overcome first.

The Problem

To handle numerous network calls, the repo server employs caching. Caching saves information closer to the user so that when needed again, it is easier and faster to retrieve. For the repo server, this means caching the response of GitHub calls so that when it needs to make the same call, it can pull from the cache instead of making a full network call.

The result of this caching needs to be stored somewhere. We didn’t want to store it locally on the machine running Argo, as that wouldn’t scale well over time. At Realtor.com, we use AWS, so our choice for file storage was Amazon Elastic File Storage (EFS). The file storage is mounted onto the repo server, and the folders and files used for caching are stored in EFS, just like any other files.

Caching and EFS are great, but they also introduced issues. The folder mounted onto the repo server for EFS held the information needed to make and maintain connections to GitHub. That information is specific to each instance of the repo server. When another replica was added, the mount used was the exact same path in EFS. This meant the information needed to make GitHub connections and the cache would be overwritten constantly by the other replicas, causing failures in cache usage and GitHub connections. EFS and shared files were great in theory but disastrous in practice.

The Solution

The requirements for the solution were clear: continue using caching and EFS. Caching was necessary because network bottlenecks would persist without it, even with scaled repo server replicas. EFS was necessary because it was easy to use and efficiently scaled the files used by caching.

The solution was not to have every repo server replica mount in the same folder in EFS. This is simple enough, but how do we do that in Kubernetes to make each mount unique? 

As of Kubernetes version 1.17, you can use supPathExpr, which allows you to specify an expression that will be used as the path to be created in your volume (in our case, EFS). Without this, the usual code is mountPath: /example/path, but with it, the expression looks like this: subPathExpr: $(POD_NAME). This expression ensures the folder created in EFS by each pod will have the same name as the pod, making every folder in EFS unique. Each replica of the repo server now gets its own folder in EFS – no more collisions!

Having individual EFS folders and no more collisions allowed the repo server to have multiple replicas. The small increase from one replica to three enabled more connections to happen simultaneously, drastically reducing deployment times in Argo.

Since Argo is already set up to use multiple instances of the repo server, increasing the number of replicas was the only additional configuration needed. This means that as more projects are added, if latency recurs, we can increase the number of replicas again to scale along with our growth and Kubernetes adoption.

To summarize, adding just one line of code resulted in a 300% improvement in deployment time. While it was an easy change, it wasn’t simple. In the future, this opens up possibilities of adding pod autoscaling (HPA), which will automatically adjust the number of replicas based on demand, further enhancing our ability to automate processes and respond quickly to engineers’ needs.

It’s amazing to see how much thought and understanding are needed for such a simple solution, highlighting the complexity and beauty of engineering.

Profile of Nathan Moore

Nathan Moore is a DevOps engineer at Realtor.com with 3 years of experience in software engineering. Nathan is also an AWS Certified Solutions Architect – Associate.

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : Realtor.com – https://techblog.realtor.com/scaling-argo-repo-server-how-increasing-replicas-solved-startup-delays

Previous Post

Israeli launches airstrikes on areas east of Khan Younis after Palestinian Islamic Jihad rocket attacks

Next Post

Why Defining Values Is Critical To Shaping How Kids Use AI

Morgan Adamski Joins PwC in Newly Created Cyber, Data & Technology Risk Division – HSToday

Morgan Adamski Leads the Charge in PwC’s Cutting-Edge Cyber, Data & Technology Risk Division

July 23, 2025
Women’s Gymnastics All Sports Schedule 2025 – Mountain West Conference

2025 Women’s Gymnastics Schedule: Can’t-Miss Matchups in the Mountain West Conference

July 23, 2025
Fyodor Konyukhov sets off on an ecological expedition across three oceans – realnoevremya.com

Fyodor Konyukhov sets off on an ecological expedition across three oceans – realnoevremya.com

July 22, 2025
‘A disaster for all of us’: US scientists describe impact of Trump cuts – theguardian.com

A Disaster for All of Us’: How Trump’s Cuts Devastated US Science

July 22, 2025
Lycoming College launches new data science academic program – NorthcentralPA.com

Lycoming College Launches Exciting New Data Science Program

July 22, 2025
Lifestyle startup Escape Plan raises $5 million to launch over 100 offline stores across India – Indian Startup News

Lifestyle Startup Escape Plan Raises $5 Million to Launch Over 100 Offline Stores Across India

July 22, 2025
Kauffman Foundation launches new executive role to lead its Real World Learning team – Startland News

Kauffman Foundation Launches Dynamic New Executive Role to Propel Real-World Learning Initiatives

July 22, 2025
Economic Planning Shouldn’t Be a Swear Word – Jacobin

Why It’s Time to Give Economic Planning Another Shot

July 22, 2025
AP Entertainment SummaryBrief at 1:51 p.m. EDT – Channel 3000

Entertainment Highlights: Key Updates You Can’t Miss

July 22, 2025
Department of Labor changes home health care, migrant farm worker rules – WBUR

Department of Labor changes home health care, migrant farm worker rules – WBUR

July 22, 2025

Categories

Archives

July 2025
MTWTFSS
 123456
78910111213
14151617181920
21222324252627
28293031 
« Jun    
Earth-News.info

The Earth News is an independent English-language daily published Website from all around the World News

Browse by Category

  • Business (20,132)
  • Ecology (734)
  • Economy (757)
  • Entertainment (21,641)
  • General (16,047)
  • Health (9,795)
  • Lifestyle (765)
  • News (22,149)
  • People (759)
  • Politics (766)
  • Science (15,974)
  • Sports (21,255)
  • Technology (15,740)
  • World (740)

Recent News

Morgan Adamski Joins PwC in Newly Created Cyber, Data & Technology Risk Division – HSToday

Morgan Adamski Leads the Charge in PwC’s Cutting-Edge Cyber, Data & Technology Risk Division

July 23, 2025
Women’s Gymnastics All Sports Schedule 2025 – Mountain West Conference

2025 Women’s Gymnastics Schedule: Can’t-Miss Matchups in the Mountain West Conference

July 23, 2025
  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

Go to mobile version