Agentic Web Scraping

Efficiently scraping information from the web using a multi-step Agentic workflow.

The problem & situation

Our client is a knowledge center on sustainability for the Dutch market. They offer a practical guide on various things that you can do today, to make a positive impact on the environment. Many of these initiatives are locally subsidized by municipalities. Not many people know this and it can be hard to find the right information on government websites. Therefore, our client offers an easy-to-use tool to the public where anyone can find if they could be sponsored by their municipality for a sustainable project. Staying on top of all sustainability related subsidies for every municipality in The Netherlands is a substantial effort and costly exercise. To make sure our client offers up-to-date information to their users, all known subsidies are regularly checked by hand.

The solution

Leveraging the latest tools in AI, we developed a multi-step Agentic workflow, to greatly reduce time and money spent on this issue. Our first Agent crawls the web and finds the location of the information we need, without human intervention. Saving us a lot of time as opposed to conventional scraping methods where we would've had to build the required logic for every website individually. Our second agent is optimized to gather the information and place it into a desired format. The last Agent in this process analyses the content we gathered and flags changes that should get human attention. There's many small steps in between to make this work, but the process is essentially simple which makes it efficient and reliable.

‍

The outcome

Using this new way of working, this entire process can be managed by just one person in a matter of minutes. Greatly reducing the cost for this particular tool and enabling our client to focus their time on more rewarding parts of their job. Additionally, accuracy and reach were both improved.

Cases

Projects that we are proud of

A selection of cases we are most proud of.

Market intelligence

Our client wanted to predict the market to make informed strategic decisions.

Marketing Optimization

Gaining insight into marketing cost and in turn optimising marketing spend to gain the most out of each campaign or activity.

Customer 360 view

The digital team wanted to have a centralised hub that would give them a 360 overview of their online and offline marketing efforts.

Harmonised Pricing Strategy

Data-driven decision making for an optimized pricing strategy.

Forecasting Engine

A headless forecasting engine using various machine learning models to find reliable outcomes at > 90% accuracy.

Data-driven impact on healthcare

Transforming raw data to actionable insights to improve digital healthcare.

AI efficiency to increase value add

Innovation, digital disruption and new, digital leadership are some keywords that describe our client.

The power of AI for summarizing meetings

We designed an AI-powered assistant to function as a powerful aid in certain situations.

Ready to take off?

Schedule a free data scan or contact us for your next challenge!

Get a free data scan Contact us