* . *
  • About
  • Advertise
  • Privacy & Policy
  • Contact
Tuesday, June 3, 2025
Earth-News
  • Home
  • Business
  • Entertainment
    Salem’s Harborwalk Garden Cultivates Community with Entertainment, Food, and Events – 105.7 WROR

    Discover the Vibrant Community Spirit at Salem’s Harborwalk Garden: A Hub for Entertainment, Food, and Fun!

    Entertainment-Focused Narrative and Culture Change Practice – New America

    Transforming Culture Through Engaging Entertainment Narratives

    Rising stars: Young classical musicians surging on social media – Yahoo

    Meet the Next Generation of Classical Music Sensations Making Waves on Social Media!

    Devin Harjes Dies: ‘Manifest’ & ‘Boardwalk Empire’ Actor Was 41 – WyomingNews.com

    Tragic Loss: Devin Harjes, Star of ‘Manifest’ and ‘Boardwalk Empire,’ Passes Away at 41

    Why Starz Entertainment Stock Soared Today – The Motley Fool

    Unpacking the Surge: What Fueled Starz Entertainment’s Stock Explosion Today!

    Unveiling the Enigmatic: First Looks at Destruction and Puck in ‘The Sandman

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Finland Set to Lead EU Quantum Technology Defense Project – IoT World Today

    Finland Set to Lead EU Quantum Technology Defense Project – IoT World Today

    AI for lawyers: Win back your time using technology – nationaljurist.com

    Reclaim Your Time: How AI is Transforming the Legal Profession

    Prosecutors accuse men of exporting U.S. military technology to China – Milwaukee Journal Sentinel

    Men Charged with Illegally Exporting U.S. Military Technology to China

    ROLAND’S NEW WIRELESS TRIGGER TECHNOLOGY, PORTER & DAVIES ON TOUR, NEW 64 AUDIO ASPIRE UNIVERSAL IEM MODELS, WAVES FREE PLUGIN PACK – Modern Drummer Magazine

    ROLAND’S NEW WIRELESS TRIGGER TECHNOLOGY, PORTER & DAVIES ON TOUR, NEW 64 AUDIO ASPIRE UNIVERSAL IEM MODELS, WAVES FREE PLUGIN PACK – Modern Drummer Magazine

    This giant microwave may change the future of war – MIT Technology Review

    Revolutionizing Warfare: The Impact of a Game-Changing Giant Microwave

    Bajeed Pattan Joins Forbes Technology Council as Innovation Leader – PRWeb

    Bajeed Pattan Takes the Helm as Innovation Leader at Forbes Technology Council!

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
  • Home
  • Business
  • Entertainment
    Salem’s Harborwalk Garden Cultivates Community with Entertainment, Food, and Events – 105.7 WROR

    Discover the Vibrant Community Spirit at Salem’s Harborwalk Garden: A Hub for Entertainment, Food, and Fun!

    Entertainment-Focused Narrative and Culture Change Practice – New America

    Transforming Culture Through Engaging Entertainment Narratives

    Rising stars: Young classical musicians surging on social media – Yahoo

    Meet the Next Generation of Classical Music Sensations Making Waves on Social Media!

    Devin Harjes Dies: ‘Manifest’ & ‘Boardwalk Empire’ Actor Was 41 – WyomingNews.com

    Tragic Loss: Devin Harjes, Star of ‘Manifest’ and ‘Boardwalk Empire,’ Passes Away at 41

    Why Starz Entertainment Stock Soared Today – The Motley Fool

    Unpacking the Surge: What Fueled Starz Entertainment’s Stock Explosion Today!

    Unveiling the Enigmatic: First Looks at Destruction and Puck in ‘The Sandman

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Finland Set to Lead EU Quantum Technology Defense Project – IoT World Today

    Finland Set to Lead EU Quantum Technology Defense Project – IoT World Today

    AI for lawyers: Win back your time using technology – nationaljurist.com

    Reclaim Your Time: How AI is Transforming the Legal Profession

    Prosecutors accuse men of exporting U.S. military technology to China – Milwaukee Journal Sentinel

    Men Charged with Illegally Exporting U.S. Military Technology to China

    ROLAND’S NEW WIRELESS TRIGGER TECHNOLOGY, PORTER & DAVIES ON TOUR, NEW 64 AUDIO ASPIRE UNIVERSAL IEM MODELS, WAVES FREE PLUGIN PACK – Modern Drummer Magazine

    ROLAND’S NEW WIRELESS TRIGGER TECHNOLOGY, PORTER & DAVIES ON TOUR, NEW 64 AUDIO ASPIRE UNIVERSAL IEM MODELS, WAVES FREE PLUGIN PACK – Modern Drummer Magazine

    This giant microwave may change the future of war – MIT Technology Review

    Revolutionizing Warfare: The Impact of a Game-Changing Giant Microwave

    Bajeed Pattan Joins Forbes Technology Council as Innovation Leader – PRWeb

    Bajeed Pattan Takes the Helm as Innovation Leader at Forbes Technology Council!

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
Earth-News
No Result
View All Result
Home Business

Building Metatron: State-of-the-Art Leads Forecasting using Transformers, Ensembling, and Meta-Learners

May 26, 2024
in Business
Building Metatron: State-of-the-Art Leads Forecasting using Transformers, Ensembling, and Meta-Learners
Share on FacebookShare on Twitter

Lead generation is our principal way to monetize on our platform in the real estate industry. Accurate lead forecasts are paramount to helping guide Sales, Product, Data Science, and Finance to set the company up for success. 

When you underestimate lead forecasts, money is left on the table because you have more lead volume you can sell. When you overestimate, you end up with under-fulfillment because you are selling more volume than is coming in. Furthermore, under-fulfillment decreases customer satisfaction, and there’s a higher likelihood of customer churn, which we vehemently want to avoid.

Previous state-of-the-art time-series models such as recurrent neural networks (RNNs) – Facebook’s Prophet – and Long-Short Term Memory (LSTM) models have worked well for us in the past. This time, we wanted to push beyond standard techniques and embark on a project that was adaptable enough to provide accurate forecasts in the face of challenges, for example, shifts from black swan events (COVID-19), rising interest rates, and market uncertainty. 

Enter Metatron

To solve this problem, we created a stacked ensemble model that combines various modeling approaches and incorporates multiple state-of-the-art temporal fusion transformer (TFT)[1] models. TFT models are effective for modeling time-series data and can be fed multiple sequential time series directly and simultaneously. The TFT model uses the transformers you have heard about in large language models (LLMs) such as ChatGPT and applies the same techniques to Realtor.com’s lead data.

TFT is a formidable approach, which we further enhance by ensembling it with traditional models. We weight each model according to its recent historical performance, optimizing our combined forecasting power. Then, we use a meta-learner to build a final stacked ensemble model, which takes in the multiple modeling approaches and outputs a single prediction per zip code and product vertical. The ZIP code predictions are made monthly and predicted out 12 months into the future. We call this innovative and adaptable approach “Metatron.” Metratron helped us overcome many challenges in the current real estate climate and make accurate predictions in the face of uncertainty. It’s also helping the business drive revenue while increasing customer satisfaction. 

This post delves further into our methodology and the various modeling approaches used to build Metatron.

Overview

To effectively manage the significant variation in lead volumes across ZIP codes, the ZIPs are categorized into deciles according to their lead volume. Lower-numbered deciles (1, 2, 3, etc.) include ZIP codes with higher lead counts, whereas higher-numbered deciles (7,8,9,10) consist of ZIPs with lower lead volumes. The low and high-volume deciles are trained separately, with deciles 1 through 9 forming one training group and the lowest-volume decile (10) forming the other. This approach allows for tailored-modeling strategies better suited to each decile group’s characteristics.

Note that we can only use decile training for models that allow passing in multiple time series. In contrast, the models we used from the Darts library work on single time series and will be referred to as local models from now on. Local models make predictions at individual ZIP code levels. We will discuss these in more detail in the following sections.

The model forecasts are output monthly, and the model training is updated monthly to ensure the most accurate predictions. Now, we will explore the traditional and various TFT models further.

Models

Local Models Using Darts

In Metatron, we leverage models from the Darts[2] library that utilize traditional and naive forecasting techniques. Each model is a local model trained on an individual time series for each ZIP code, creating a distinct model for each product and every ZIP code. Total models=# zip codes x # products, where each model has a single prediction. 

These models include:

Naive Drift: A model that constructs a straight line from the initial to the final data point in the training series and extends this trend into the future.

Naive Mean: This model consistently forecasts the average value of the training series.

Naive Moving Average: This model employs an autoregressive moving average (ARMA) approach with a specified sliding window parameter.

Naive Seasonal: This naive model forecasts based on the value from K time steps prior. In our application, we use K=12, reflecting the assumption that the patterns observed in the previous 12 months will recur in the following 12 months.

Temporal Fusion Transformer 

The other key piece of the puzzle in Metatron is the transformer-based model known as the TFT. The “Temporal” aspect of TFT highlights its focus on time-related or sequential data with inherent dependencies (each product and ZIP code’s monthly lead volume). “Fusion” signifies its capability to amalgamate information from various data sources. This can be historical time-varying features, which are only known up to the time of prediction (for example, the past lead volume of all product verticals), static or time-invariant features (for example, ZIP or latitude/longitude), and known future inputs (for example, months or holidays like Christmas day). “Transformer” denotes its basis in the Transformer architecture that allows it to learn longer-term dependencies [1] and is the same architecture as the basis for the current LLM revolution.

TFT is a global model that analyzes the relationships between various time series of ZIP codes collectively rather than individually. It utilizes a shared representation to uncover underlying patterns and trends across all series. By modeling the combined distribution, TFT can identify complex patterns and relationships that are difficult or nearly impossible for local models to extract, even if expertly designed.

In addition to being a global model, TFT enables simultaneous training and prediction of multiple targets. This means you can use a single model to predict not only across every zip code but also for every product – at once. Unlike local models, which require separate inputs for each product, you can input each product’s time series (Connections Plus, MarketVIP, Ready Connect Concierge, and Third Party) into a single model and obtain distinct forecasts for each product (multi-target).

We tried multi-target TFT models, which were predicted using a single model across all products, and single-target models, which independently predicted each product. The single-target models used the same data as the multi-target models (Connections Plus, MarketVIP, Ready Connect Concierge, and Third Party time-series data) but just had that single product’s predictions output for each model. There were tradeoffs between using a multi-target and single-target model from modeling, ease of implementation, and results standpoints.

Multi-target TFT Model vs. Single-target Model Tradeoffs

Multi-target had only one set of tuned hyperparameters and one model to run during inference, simplifying the code and infrastructure required for training.

Single-target allowed for a simpler optimization function and generally quicker training times but required more models with different optimal hyperparameters.

For both the multi-target model and single-target model, we trained on deciles 1-9 and decile 10 separately. Therefore, the multi-target had a total of 2 models equal to the number of decile groups, where each model has a prediction for each ZIP in that decile group across all the products simultaneously. The single-target model had a total of 8 models equal to the number of decile groups (2) times the number of products (4), where each model has a prediction at the product level for each ZIP in that decile group.

Weighted Ensemble on Recent Performance

Following the development of the TFT and naive models in our Metatron framework, we refined our forecasting accuracy through a targeted ensemble methodology. This approach involved weighting models based on their recent performance, specifically over the last three months, using  Weighted Average Percentage Error (WAPE) evaluated at the decile level. We prioritized models with lower error rates and calculated the reciprocal of each model’s WAPE. These reciprocal values were normalized to determine each model’s proportional contribution to the ensemble. This ensured that weights were uniquely tailored to each decile. 

Finally, we applied these weights to the predictions within each decile, multiplying each ZIP code’s predictions by the corresponding decile weight and summing the results to generate the final ensemble output. This was all computed for each product individually and not globally across all products. This methodology leverages the strengths of both global and local models and dynamically adjusts to the most recent data, optimizing prediction accuracy across our vast array of ZIP codes and products.

Introducing Metatron: Stacked Ensemble Model (Catboost – Meta-Learner)

Now, we have all the necessary pieces to unveil the full power of Metatron, our CatBoost[3] meta-learner that combines all the efforts presented above. This model integrates predictions from global models (TFT models), local models (Darts models), and the decile-weighted ensemble model. It also incorporates additional time-related contextual data such as year, yearly trends, and cyclic time features. The diagram below illustrates how the final Metatron predictions are generated:

The base models feed into a WAPE-weighted ensemble model, and then all models, including the WAPE-weighted ensemble model, feed into Metatron, a Catboost meta-learner. Also, the diagram shows if the models were trained jointly or separately by product and trained at the zip-level or decile level. To get the total number of models in each block outputting predictions for each zip code and product, you need to multiply the two numbers in parentheses (x) of the dotted color boxes for that corresponding block. For example, TFT Single-target would be (4) x (2), so 8 total models giving predictions across all zip codes and products.

Furthermore, the meta-learner was trained differently for low and high-volume decile groups to optimize performance better:

Mean Absolute Error (MAE) was used as the loss function for higher volume groups (deciles 1-9) due to its robustness for larger and non-zero biased datasets.

Tweedie loss function was chosen for the lowest volume group (decile 10) to handle scenarios with high incidence of zeros effectively.

The meta-model outperforms all other models to effectively piece together accurate ZIP-level predictions across the various products. It does this by synthesizing insights from the various models and additional features given to the meta-model. This was the final piece of the modeling puzzle to get the desired adaptable state-of-the-art results.

We evaluated the performance of our models using several metrics, including total percentage error, the percentage of ZIP codes with errors within 20%, and WAPE. WAPE helps us evaluate how accurate each model is by measuring the size of errors in relation to actual values. It’s calculated simply by taking the total of all absolute errors between predicted and actual values, dividing that by the sum of all actual values, and then converting that number into a percentage.

This metric is particularly useful for comparing performance consistently across different ZIP codes and deciles, ensuring we understand which models perform best in which areas.

When comparing model performance at both the ZIP code level and the aggregate decile level, Metatron, our Catboost stacked ensemble model, consistently stands out as the most accurate and reliable model.

ZIP Code Level Analysis:

Metatron excels with 60.95% of ZIP codes within the ±20% error threshold, and it captures 64.54% of total actual leads within this range, as shown in the figure below. The Current Champion and TFT models have around 47% ZIP codes within the ±20% threshold. Therefore, the average percent relative accuracy increase with Metatron at the zip code level is 30%. This indicates Metatron’s superior prediction accuracy over the current champion or TFT models by themselves.

Furthermore, we see this increased accuracy stable over month-to-month predictions. The plot below shows the prediction for the month after the plot above. Metatron again excels, with 60.46% of ZIP codes within the ±20% error threshold and 62.89% of total actual leads within this range captured. For this month, one of the TFT models dropped to 34.32% of ZIP codes with the ±20% error threshold from 47%, highlighting the higher month-to-month volatility in individual models. Notably, this month’s average percent relative accuracy increase is 42%. Once again, Metatron provides vastly improved relative prediction accuracy, and this trend is expected to continue in future months not shown here.

Aggregate Decile Level Analysis

Metatron maintains the lowest WAPE across all deciles, highlighting its superior performance. The Naive Seasonal model shows the highest WAPE, indicating the lowest accuracy. Other models, including the Current Champion, TFT Multi-Target, TFT Single-Target, Naive Ensemble, and Decile Ensemble, consistently have higher WAPE values than Metatron.

Higher-volume ZIP codes (deciles 1 and 2, for example), which have better data distributions and less noise, benefit more from single-target models. In contrast, lower-volume ZIP codes, which are noisier and sparse (containing many zeros), benefit from the joint optimization across all product verticals provided by multi-target models. The plot below shows that the single-target model performs better in the lower deciles, and the multi-target model performs best in decile 10.

Overall Performance

Metatron stands out as the most reliable model and has demonstrated superior accuracy, minimizing the width of the error distribution at the ZIP code level and lowering WAPE at the decile level. This reinforces Metatron’s effectiveness in utilizing predictions from individual models to enhance the overall performance across all ZIPs. Similar results were observed across different forecasting periods, making it the preferred model and new champion. Long live Metatron!

Lead generation is paramount to a company’s success, particularly in the real estate industry. We embarked on a journey to get better and better at predicting future lead forecasts and developed a robust, state-of-the-art approach – Metatron – to generate lead forecasts. Metatron has a basis in modern transformer networks using TFT combined with many other modeling techniques, such as ensembling, meta-learning, and innate knowledge about how the real estate market works, to deliver accurate monthly forecasts that predict 12 months into the future. 

Metatron enables greater revenue by not underpredicting lead volume, which can result in money left on the table for Sales. At the same time, we can delight customers by delivering on our promise and not selling too much lead volume in the market. This provides Realtor.com with more accurate lead forecasting abilities and a competitive edge despite market uncertainties.

Please contact Jeff Spencer or Siddharth Arora with any comments or questions. We are always interested in talking to others in the industry.

Ready to reach new heights at the forefront of digital real estate? Join us as we build a way home for everyone.

Lim, Bryan, Sercan Ö. Arık, Nicolas Loeff, and Tomas Pfister. “Temporal fusion transformers for interpretable multi-horizon time series forecasting.” International Journal of Forecasting 37, no. 4 (2021): 1748-1764.

Herzen, Julien, Francesco Lässig, Samuele Giuliano Piazzetta, Thomas Neuer, Léo Tafti, Guillaume Raille, Tomas Van Pottelbergh, et al. “Darts: User-friendly modern machine learning for time series.” Journal of Machine Learning Research 23, no. 124 (2022): 1-6.

Dorogush, Anna Veronika, Vasily Ershov, and Andrey Gulin. “CatBoost: gradient boosting with categorical features support.” arXiv preprint arXiv:1810.11363 (2018).

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : Realtor.com – https://techblog.realtor.com/building-metatron-state-of-the-art-leads-forecasting-using-transformers-ensembling-and-meta-learners

Tags: buildingbusinessMetatron
Previous Post

The Seven Archetypes Of Leadership And How Entrepreneurs Become CEOs

Next Post

Rahul Kohli Talks Missing “Fantastic 4”

Plan to control nonpoint sources of pollution – Washington State Department of Ecology (.gov)

Taking Charge: Strategies to Tackle Nonpoint Source Pollution in Washington State

June 3, 2025
The Uncertain and Shifting Future of Ph.D.s in Science – Undark Magazine

Navigating the Unpredictable Future of Science Ph.D.s: Challenges and Opportunities Ahead

June 3, 2025
‘Sport Science’ host John Brenkus dies after battle with depression – ABC News

‘Sport Science’ host John Brenkus dies after battle with depression – ABC News

June 3, 2025
Cost of ‘minimum retirement lifestyle’ has fallen amid lower energy prices – London Evening Standard

Affordable Retirement: How Lower Energy Prices Are Redefining Minimum Lifestyle Costs

June 3, 2025
Opinion | Poland Just Sent an Ominous Signal to the World – The New York Times

Poland’s Alarming Message to the Global Community

June 3, 2025
Report ranks Iowa as state with the worst economy – KCRG

Report ranks Iowa as state with the worst economy – KCRG

June 3, 2025
Salem’s Harborwalk Garden Cultivates Community with Entertainment, Food, and Events – 105.7 WROR

Discover the Vibrant Community Spirit at Salem’s Harborwalk Garden: A Hub for Entertainment, Food, and Fun!

June 3, 2025
18 High-Protein Breakfasts for Better Gut Health – EatingWell

Boost Your Gut Health with These 18 Delicious High-Protein Breakfasts!

June 3, 2025
Diagnosis for 6.2.25: Checking the pulse of Florida health care news and policy – Florida Politics

Diagnosis for 6.2.25: Checking the pulse of Florida health care news and policy – Florida Politics

June 3, 2025
Finland Set to Lead EU Quantum Technology Defense Project – IoT World Today

Finland Set to Lead EU Quantum Technology Defense Project – IoT World Today

June 3, 2025

Categories

Archives

June 2025
MTWTFSS
 1
2345678
9101112131415
16171819202122
23242526272829
30 
« May    
Earth-News.info

The Earth News is an independent English-language daily published Website from all around the World News

Browse by Category

  • Business (20,132)
  • Ecology (662)
  • Economy (676)
  • Entertainment (21,582)
  • General (15,258)
  • Health (9,719)
  • Lifestyle (679)
  • News (22,149)
  • People (677)
  • Politics (685)
  • Science (15,897)
  • Sports (21,181)
  • Technology (15,663)
  • World (664)

Recent News

Plan to control nonpoint sources of pollution – Washington State Department of Ecology (.gov)

Taking Charge: Strategies to Tackle Nonpoint Source Pollution in Washington State

June 3, 2025
The Uncertain and Shifting Future of Ph.D.s in Science – Undark Magazine

Navigating the Unpredictable Future of Science Ph.D.s: Challenges and Opportunities Ahead

June 3, 2025
  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

Go to mobile version